All Questions
Tagged with floating-point c#
67
questions with no upvoted or accepted answers
4votes
0answers
384views
C# Change FPU rounding mode
I'm attempting to write an interval arithmetic library in C# .NET, but in order to do this accurately I need to be able to control the rounding mode of floating point operations.
After a bit of ...
- 639
4votes
2answers
1kviews
How to correctly pass a float from C# to C++ (dll)
I'm getting huge differences when I pass a float from C# to C++.
I'm passing a dynamic float wich changes over time.
With a debugger I get this:
c++ lonVel -0.036019072 float
c# lonVel -0....
3votes
1answer
146views
What guarantees does System.Numerics.Vectors provide about size and bit order?
I have implemented a vector-based c# approximation of Log. It includes unsafe code. It's been working fine in a number of environments, but on a recent deployment has fallen over. The implementation ...
- 101
3votes
1answer
312views
Is it safe to cast Math.Round result to float?
A colleague has written some code along these lines:
var roundedNumber = (float) Math.Round(someFloat, 2);
Console.WriteLine(roundedNumber);
I have an uncertainty about this code - is the number ...
- 124k
3votes
0answers
196views
Calculating floating point error bound
I have geometrical algorithms and im struggling with floating point inaccuracies.
For example, I'm calculating wether a point lies on the left/right/on a plane (C#):
const double Epsilon = 1e-10;
...
- 610
3votes
2answers
1kviews
Incorrect float subtraction result
DataSet
id qty CheckIn CheckOut
5 10 1 0
5 10 0 1
5 1.6 1 0
5 0.4 0 1
5 0.4 0 1
5 0.4 0 1
I am trying to ...
- 229
2votes
1answer
126views
Double values comparison w/ tolerance for "point" equality in a DXF drawing
I have a simple algorithm that fails sometimes because it is comparing doubles. I look at a DXF drawing and get all the line segments, and also break it down into a series of points. When looping ...
- 141
2votes
0answers
40views
Casting long.MaxValue -> float -> long results in overflown value
We noticed quite a problem in our codebase lately which caused our RNG to go bonkers. Currently all our game variables are saved as floats, among them the max long value from the game server. When ...
2votes
0answers
657views
What MySQL "DECIMAL" precision and scale is required to store a C# decimal value?
I want to store some decimal values in my MySQL database. The obvious MySQL database type for that would be DECIMAL I guess.
What precision "a" and scale "b" do I have to set for DECIMAL(a,b ) ...
- 5,504
2votes
3answers
3kviews
32-bit Grayscale Tiff with floating point pixel values to array using LibTIFF.NET C#
I just started using LibTIFF.NET in my c# application to read Tiff images as heightmaps obtained from ArcGIS servers. All I need is to populate an array with image's pixel values for terrain ...
- 21
2votes
1answer
141views
How to fix floating point artifacts in StdDev calculation?
I trying to calculate standard deviation with a next method:
private static double? StdDev(IReadOnlyCollection<double> items) {
if(items == null) {
throw new ArgumentNullException("items")...
- 1,469
2votes
0answers
1kviews
Show Amount upto two decimal places without rounding off in DataGrid
Say for example i have a large number 12345678.88 right now when i display it in the Datagrid
with the use of
<asp:Label ID="txtlineAmt" runat="server" ReadOnly="true" Text='<%# Eval("...
- 4,442
2votes
2answers
272views
hashing a floating point number for test case purposes in the .Net runtime
I have complex test cases with geometric entities. It is easy for me to visually verify that the test case has passed by looking at the generated geometry in a viewport like so.
When I am satisfied ...
- 385
2votes
5answers
2kviews
Join two strings into a single decimal value
I want to get a decimal value, for example 500.45, from two strings. How to achieve this operation?
decimal value; //like 500.45
string txt1, txt2;
txt1 = "500";
txt2 = "45";
value = Convert....
- 1,348
1vote
1answer
74views
Strange double subtraction result in .NET
I'm aware of the inaccuracies of floating point operations but the results of the below code seem rather odd to be. Can anybody please explain it?
This is with .NET Framework 4.8 on Windows 10. The ...
- 300
1vote
0answers
126views
Higher precision floating point calculations
I would like to find a way to make higher precision calculations in a compute shader using multiple floats to represent a single number.
My goal is a mandelbrot renderer where you can zoom deeper ...
1vote
0answers
46views
Is it possible to manually increase the precision of a double as a bit string through calculations
In order to create a floating point number from a decimal value, we need to have a sign bit, and exponent, and a mantissa.
Double precision uses 11 exponent bits, and 52 mantissa bits.
Let's say we ...
1vote
0answers
74views
Puzzled over Float vs Double benchmark
After searching SO for performance differences between floats and doubles in .NET, I put together a simple benchmark to test 2 methods, each with floats and doubles, debug and release. After running ...
- 1,627
1vote
0answers
83views
How do I set the default decimal number separator in Python?
I am working on two applications, one running in C#/.NET and the other with the Python 3 interpreter targeting Windows 10 only. The two applications communicate with each other using a local network ...
- 3,625
1vote
0answers
63views
Do .NET floating-point operations agree between x64 debug and release?
In this question, the asker wondered whether or not one could expect floating point operations to behave similarly between debug and release configurations.
The answers given there show that this is ...
- 15.7k
1vote
0answers
246views
using float.Parse() vs casting to get a value from double
I wasn't able to find a question that directly answers this question, I came across this line of code in a project at work (I added the comment)
// act.score is a double
float score = float.Parse(act....
- 8,497
1vote
2answers
148views
Reversible decimal to double conversion
I'm curious to know if you could convert decimal to double, but still be able to come back to the same decimal I started with. Of course, assuming you use 15 or less significant digits. Basically, can ...
- 3,621
1vote
1answer
1kviews
C# Decimal type rounding off inappropriately
I've got an in house banking solution for a summer camp that I'm a part of. The issue is coming up that some accounts are exactly a penny off at the end of the season.
I'm using SQL Server Express, ...
- 2,302
1vote
0answers
69views
Formatting numbers in vertically aligned textboxes in C# with the floating point also vertically aligned
On my UI, there are several textboxes that are vertically aligned. I'd like to show the numbers in them in a way that all floating points are also vertically aligned. I also need to show the sign of ...
- 889
1vote
1answer
179views
Could Mono.Simd or Microsoft.Bcl.Simd provide deterministic floating point arithmetic to C#?
C# cannot be coerced into deterministic floating point arithmetic, the designers seem to have considered the performance overhead too significant.
I'm trying to assess whether it would be possible to ...
- 4,110
1vote
0answers
557views
Converting 4-byte OLE (VB6) to double/float in C#
Well, I have a 4 (four) bytes array that I need to convert to double data-type (or float) in C#.
I believe that the 4 bytes array is an OLE Automation one.
Tried the following, but no success:
// ...
- 109
1vote
1answer
457views
Extjs float value is being passed as null in c# controller
I have a float value in extjs model. For some reason any value I enter and pass to c# controller, its value is always null. I checked the json data being pass and I can see the passed value.
This is ...
- 1,357
1vote
0answers
147views
Byte[] to float conversion issue
I have a binary file written in big endian enviorment using c code.
er
On my PC a C# application is reading bytes from the file using BinaryReader and converting it to Managed struct using Marshalling....
- 247
1vote
2answers
578views
Floats changing when passing them from CLI to C#
EDIT : i wrote the same code that i had at work at home to recreate the bug. this is the exact code !!
I'm trying to pass floating number from c++ to c# and i'm using c++ CLI as intermediate.
I have ...
- 4,420
1vote
2answers
483views
C#: Store number in database with 2 places of decimal. Eliminate unwanted numbers
I have a field NumberValue1 declared like this
public double NumberValue1 { get; set; }
NumberValue1 has a datatype of Number in the
oracle database
I read in a value from an excel file which is
22....
- 1,657
1vote
0answers
2kviews
Getting significand and exponent value from floating point literal with C#
A literal of 0.25 can be expressed with 1*2^(-2) so that we can get 1 as significand and -2 as exponent.
What function can I get the significand/exponent with C#?
int significand = GetSignificand(0....
- 167k
1vote
2answers
47views
Convert Long to Float Changing the number c#
I have long vairable long x = 231021578;
and when I convert it to float like float y = x;
the value of y will be 231021584
I want to know why this happen. Float is stored in 4 bytes and its range is ...
- 17
1vote
2answers
2kviews
Error when trying to save data to a database with dbcontext. Not a valid instance of type float
I cant seem to figure this out. The error I get is
"The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is >incorrect. Parameter 13 ("@10"): The supplied value is ...
- 697
0votes
0answers
55views
Need a .NET Framework solution for IEEE-754 float/double parse and ToString
.NET framework does not give me expected results for float/double parsing and ToString. It is fixed in .NET 6 (even in .NET Core 3.0). However, I cannot migrate my whole project from .NET framework 4....
- 34
0votes
0answers
61views
.NET 6.0 vs .NET Framework Number Rounding
As I was migrating a complex .NET Framework project to .NET 6.0, I noticed different results under some circumstances. I was able to trace down the issue to a basic inequality involving floats and a ...
- 338
0votes
0answers
33views
I am having trouble evaluating the height in c#, when i introduce a number smaller than 1.5 still says come in
using System;
namespace Tarea_C_
{
class Program
{
static void Main(string[] args)
{
Console.Write("Write your age: ");
int age = Convert....
0votes
0answers
58views
Is there a way to tell the compiler to compile with "fast-math" or something similar in C#?
In some other languages, you can allow the compiler to perform some floating point optimizations, one of which is rewriting your code into algebraically equivalent expressions. Here are some examples ...
- 122
0votes
0answers
54views
How to insert value as 'null' instead of 0 to numeric column in sql using c#?
I need to insert float value as 'NULL' for all columns in sql instead of 0.00000 or 0.I have written code as below.Is there any better alternative for this since I should do this for all columns ...
- 27
0votes
0answers
63views
C# - Converting data from Decimal to float is truncating after decimal points while retrieving data from database
I am trying to fetch percentage value from database and my database query gives 2 digits after decimal, but when i am trying to access that value in c# code, the decimal point and the 2 digits after ...
- 201
0votes
0answers
132views
Why does float.TryParse() accepts "1.000e-45" and rounds it to 1.401298E-45?
I try to validate that an input string is a float. To achieve this I pass the input string like "1.000e-45" which is closer to 0 than the c# float minimum allows (1.401298E-45). I expect ...
- 2,012
0votes
1answer
36views
Cutting corners with float-to-string representation?
I have a float I need to turn into a string with 5 decimals precision (X.XXXXX), which means I need to have at least 6 decimals for round up/down. The issue is that the operation to get integer ...
- 11
0votes
0answers
80views
Why .net C# float is truncated to thousand in string representation?
How come the following is possible ?
immediate console:
"{0:F20},{1:F20}, {2:F20}".format(
res_ts, // float
AppMgr.logon_info.res_ts, //float
res_ts - AppMgr.logon_info.res_ts
)
"{0},{...
- 946
0votes
0answers
106views
How to find the similarity (as a percentage) between two float[] arrays in c#
I need to get the similarity between two float[] arrays of the same lengths and return have this similarity be returned as a float which will be the percentage. How would you do this in c#?
0votes
1answer
485views
Is it valid check for infinity with the equals operator?
c# how to check if a number is infinity
Answer: Double.IsInfinity, Double.IsPosiviteInfinity, etc.
So now comes my question: Is it valid to simply do:
double d = Double.PositiveInfinity;
if(d == ...
user4385532
0votes
0answers
58views
How to prevent Floating Point Excel Data loaded with SQL to be truncated
When I load floating point data from Excel in my C# software using SQL the data ends up being truncated.
For example, instead of 0,054034 I end up with 0,054.
The relevant part of my code is:
...
- 31
0votes
1answer
82views
More elegant way to determine whether two floats have same signs in C#?
Is there any way that can compare the signs of the two floats quickly in C#? I doubt the performance of comparing a float with 0 again and again. However, it seems that bit operation of float is ...
- 13
0votes
0answers
58views
Converting floats to ints reliably
I have an application that uses floats to handle distances, for example, 2.5f is 2.5 meters (250 centimeters). Now I want to interface with that application without using floats via network. I can't ...
- 872
0votes
0answers
84views
Taking a Floating point number to 30th decimal point in C#
I have a number which is 0.010000000000000000000000000001. I want this number to be stored as it is and want to increment to it like 0.010000000000000000000000000002
0.010000000000000000000000000003
0....
0votes
1answer
56views
IEEE floating point addition
instead of
float a = 32.342 , b = 193.132
float total = a + b
How can I convert them to 32 bits and add using their 32 bits explicitly?
- 727
0votes
0answers
258views
Float Precision in the Visual Studio Immediate Window
Undertaking a code review I have come across some strange behaviour I don't understand in the C# immediate window.
all the variables below are floats and calculate the gradient of a line. ...
- 2,950