288 precision questions.

How to correct not precise values

1 answers, 37 views list-manipulation import precision
I have the source file file = "test.txt" with these values: ...

FullForm[1. - (0.3 + 0.4)] gives 0.30000000000000004 [duplicate]

0 answers, 53 views error precision
I had a serious error propagation in my computations (Mathematica 10.4.1 Windows 64 bit). After some investigation, I realized the problem comes from something like this: FullForm[1. - (0.3 + 0.4)] ...

1 Rounding numbers to certain numbers of significant figures

2 answers, 49 views precision output
I have code which outputs numbers, for example,(0.2945847, 0.8647834, 1.6*10^-6) and similar. I would like to take these numbers and round them to 4 significant figures, but I can't find a way to do ...

Identical expressions subtracted from each other giving small but significant errors

0 answers, 29 views list-manipulation precision
In my current code, I have formed a list of numbers with two different methods. For the first method, I wrote ...

2 How can I remove the FindRoot error from my calculations?

3 answers, 237 views plotting equation-solving precision
If you run my calculations given below ...

Precision for function

I have a notebook where I have to work with high precision (say, 100). I have tried several methods to define it for the entire notebook, including ...

Computing a sum with high precision

2 answers, 51 views numerics precision numerical-value
I am aware that a similar question has been asked, for example, here and here, but none of the proposed solutions solves my problem, so please wait before you mark this question as a duplicate. ...

1 Performance Tuning - How can I make Mathematica to use less than certain digits number?

I would like to know if there is a way to make Mathematica to use at most certain digit number. For example, Suppose I want to calculate 2*Pi.x=N[Pi] y=2*x...

3 SetAccuracy not resulting in duplicates for GeoData like SetPrecision does

1 answers, 31 views precision accuracy maps
I am continuing to work with GPS data related to trail routes and am trying to optimize the code by removing route points from a GPX file that are very close to each other by setting the accuracy of ...

1 Precision error in FindMinimum

I am trying to solve a system of three non-linear equations for three unknowns (e1, e2, phi). I am using FindMinimum to solve the system (I am minimizing the norm ...

3 MinRecursion changes result of integration by 100 orders of magnitude

1 answers, 61 views numerical-integration precision
I want to compute a triple integral. I did it using two codes: the first is ...

5 Precision-related error in Audio

1 answers, 65 views bugs precision sound machine-precision
Bug introduced in 11.3 or earlier CASE:4075745 I have been experimenting with using Audio and associated functions for data acquisition using an external sound ...

3 How to avoid density plot being too ragged?

1 answers, 58 views plotting precision
I have a density plot as below:-v = {2, 3}; w = {4, 6}; DensityPlot[Exp[-w.Abs[v - {x, y}]], {x, 0, 5}, {y, 0, 7}]As you can see, the boundary of the white ...

1 How does one specify numerical precision for FortranForm[] and CForm[]

1 answers, 32 views output-formatting precision
I have a long function composed of polynomials and exponentials that I would like to convert to an appropriate form to cut and paste as an equation into SolidWorks. FortranForm gets me close and with ...

3 WorkingPrecision in NDSolve causes failure when solving a simple PDE

I have fivefold multiple integral and I wanted a speed calculations. I came across on this Question and had already begun the problems. Here is a toy example with simple double integral: ...

4 SetPrecision and setting precision with a backtick

3 answers, 95 views numerics precision
When you use SetPrecision to set the precision of say 1.1 you will get some seemingly random numbers padded before the rest will ...

Eigenvalues not invariant after similarity transformation

I have two examples of matrices that undergo a similarity transformation: a random matrix, and an exclusion-process inspired matrix. The similarity transformation $\sigma$ is given by an $L \times L$ ...

4 How to force precision in region plots containing Erfc

2 answers, 144 views plotting numerics precision
I'm trying to use regionplot for functions that involve Erfc. A slightly contrived example follows, but it illustrates my difficulties. The command ...

1 How can I improve the numerical results while working Mathematica

2 answers, 53 views equation-solving numerics precision
I'm trying to solve nonlinear equations using Newton-type methods with very high accuracy using Mathematica. I found many research papers in which the numerical results are calculated with very high ...

1 Numerical error in compile, Strange behaviour in numerical evaluation

First of all, apologies for the large numbers, I couldn't find a working example for a far lower order and the zeros are necessary... my problem is this: I have a large polynomial (from high-order ...

8 Contour integration, residues and precision of the poles

1 answers, 141 views calculus-and-analysis complex precision
Task I am trying to evaluate the contour integral of some functions. To have a concrete example, let's use $$f(z,s) = \frac{1+(4+2s)\, z}{z - \left( 9 + 35s + 24s^2 + 4s^3 \right) z^2 + 8z^3}$$<...

6 How to get accurate solutions from FindRoot?

2 answers, 255 views equation-solving precision

5 What is a sufficient decrease in the merit function?

1 answers, 255 views equation-solving error precision
As far as I understand, the merit function measures how far the data is from the fitted function. I am calulcating the second partial derivative of a function, and then numerically finding its maxima ...

5 Precision loss in FourierDCT

2 answers, 193 views fourier-analysis precision
I have high precision data that I want to do Fourier transform with, but it looks like the precision is lost dramatically. ...

Precision issue using functions

1 answers, 29 views functions precision
I just want to get a smooth plot but the calculated values are "discrete" which seems to be a precision problem. I tried N[],SetPrecision[],SetAccuary[] without any success. So I paste the Code in its ...

Iterative calculation of a number-theoretical constant [closed]

2 answers, 117 views precision number-theory
Recently the decimal expansion of a number theoretic constant was searched for, which is the analog of the Landau-Ramanujan constant in a certain context. The constant starts 0.638909... It can be ...

1 Interpolation is not smooth

Consider a function ...

1 Printing out reals in a table with a specified number of digits

I would like to print out an list of reals in a table with perfectly identical number of digits so that they can be perfectly right-aligned. However, once I convert them to strings, ...

1 Plotting with small variations along the vertical axis

3 answers, 64 views plotting precision
I can't see the small changes of my function when I plot it. What can I do? ...

3 digits of NDSolveValue

I am solving PDE by means of NDSolve and I am interested in a highly accurate value for the solution at a certain point. For some reason, the solution obtained by ...

High precision NDSolve

0 answers, 54 views differential-equations precision
I need to solve an ODE with very high precision. I am not sure yet how much precision I need so I have some question before changing the settings. From what I understand, the working precision is how ...

2 How to avoid problems with the large imaginary part of LegendreQ?

I am trying to evaluate sums involving LegendreQ[n,x], the n-th order Legendre function of the second kind $Q_n(x)$, at large $x>1$. However, the problem is that ...

1 WorkingPrecision and AccuracyGoal for using with NIntegrate

I'm modeling a quantum tunneling barrier. Because I want to express my quantities in mks units, that means that some of the variables I'm using are pretty small (10^-9). I use NIntegrate for part of ...

4 Mathematica precision common problems

1 answers, 127 views precision arbitrary-precision accuracy
When I do something a little bit more complicated than standard documentation examples, I often hit precision problem. Or I accidentally disprove Riemann hypothesis Like this ...

5 Is FindRoot wrong about its WorkingPrecision?

2 answers, 511 views equation-solving precision
EDIT: I filed a bug report and after a small back-and-forth the support person agreed this is a bug. He said: "I have filed a report with the developers to issue warning messages when the arithmetic ...

7 How to round half down in Mathematica?

2 answers, 352 views numerics precision
I found this and this on this StackExchange as to why Mathematica rounds half to even, which means thatRound[{0.5, 1.5, 2.5, 3.5, 4.5}]gives ...

1 Precision Issue with Log

1 answers, 46 views precision machine-precision
I have the following codes. Naturally, I expect to get zero. I thought it is due to some possible issues stated as "Machine-precision inputs can give numerically wrong answers on branch cuts:". ...

61 When can I assume that all decimal digits returned by Mathematica are provably correct?

How to Control the Precision and Accuracy of Numerical Results Arbitrary-Precision Numbers Mathematica works with exact numbers and with two different types of approximate numbers: machine-...

how to avoid set precision for the components of an array? [duplicate]

I have an array a with 9 components and when i set precision for this array, components of a also get the precision.however, i want to avoid that and only the number get the precision. for instance, a[...

37 Is manual adjustment of AccuracyGoal and PrecisionGoal useless?

This is a problem confusing me for years.AccuracyGoal and PrecisionGoal are two options that I never truly understand and, to ...

6 Is it possible to generate a message when precision upgrade occur?

I use Mathematica mostly for numerical simulation, so most of the time double precision (or machine precision) is enough for me. Mathematica has this nice feature of automatically upgrade to arbitrary ...

29 Why does Mathematica results differ from C++ results within machine precision?

2 answers, 3.696 views precision machine-precision c++
I am trying to wrap my head around floating-point precision. Performing the following numeric operation:275./6.*1.03692775514337in Mathematica and C++ and ...

How to use Working Precision to get good result and plot?

2 answers, 202 views plotting precision machine-precision
I have a function f(x) which I would like to see going up for very low values of x: from an asymptotic study, I expect it to go to Infinity for low x; however, I can see an elbow and many oscillations....

8 What explains the inaccuracies in machine-precision 'integers' with trigonometry or powers?

1 answers, 462 views precision machine-precision
This Mathematica tutorial notes some strange behaviour with calculations such as Sin[10^50]: ...