# All Questions

Tagged with floating-point c#

67
questions with no upvoted or accepted answers

4votes

0answers

384views

### C# Change FPU rounding mode

I'm attempting to write an interval arithmetic library in C# .NET, but in order to do this accurately I need to be able to control the rounding mode of floating point operations.
After a bit of ...

4votes

2answers

1kviews

### How to correctly pass a float from C# to C++ (dll)

I'm getting huge differences when I pass a float from C# to C++.
I'm passing a dynamic float wich changes over time.
With a debugger I get this:
c++ lonVel -0.036019072 float
c# lonVel -0....

3votes

1answer

146views

### What guarantees does System.Numerics.Vectors provide about size and bit order?

I have implemented a vector-based c# approximation of Log. It includes unsafe code. It's been working fine in a number of environments, but on a recent deployment has fallen over. The implementation ...

3votes

1answer

312views

### Is it safe to cast Math.Round result to float?

A colleague has written some code along these lines:
var roundedNumber = (float) Math.Round(someFloat, 2);
Console.WriteLine(roundedNumber);
I have an uncertainty about this code - is the number ...

3votes

0answers

196views

### Calculating floating point error bound

I have geometrical algorithms and im struggling with floating point inaccuracies.
For example, I'm calculating wether a point lies on the left/right/on a plane (C#):
const double Epsilon = 1e-10;
...

3votes

2answers

1kviews

### Incorrect float subtraction result

DataSet
id qty CheckIn CheckOut
5 10 1 0
5 10 0 1
5 1.6 1 0
5 0.4 0 1
5 0.4 0 1
5 0.4 0 1
I am trying to ...

2votes

1answer

126views

### Double values comparison w/ tolerance for "point" equality in a DXF drawing

I have a simple algorithm that fails sometimes because it is comparing doubles. I look at a DXF drawing and get all the line segments, and also break it down into a series of points. When looping ...

2votes

0answers

40views

### Casting long.MaxValue -> float -> long results in overflown value

We noticed quite a problem in our codebase lately which caused our RNG to go bonkers. Currently all our game variables are saved as floats, among them the max long value from the game server. When ...

2votes

0answers

657views

### What MySQL "DECIMAL" precision and scale is required to store a C# decimal value?

I want to store some decimal values in my MySQL database. The obvious MySQL database type for that would be DECIMAL I guess.
What precision "a" and scale "b" do I have to set for DECIMAL(a,b ) ...

2votes

3answers

3kviews

### 32-bit Grayscale Tiff with floating point pixel values to array using LibTIFF.NET C#

I just started using LibTIFF.NET in my c# application to read Tiff images as heightmaps obtained from ArcGIS servers. All I need is to populate an array with image's pixel values for terrain ...

2votes

1answer

141views

### How to fix floating point artifacts in StdDev calculation?

I trying to calculate standard deviation with a next method:
private static double? StdDev(IReadOnlyCollection<double> items) {
if(items == null) {
throw new ArgumentNullException("items")...

2votes

0answers

1kviews

### Show Amount upto two decimal places without rounding off in DataGrid

Say for example i have a large number 12345678.88 right now when i display it in the Datagrid
with the use of
<asp:Label ID="txtlineAmt" runat="server" ReadOnly="true" Text='<%# Eval("...

2votes

2answers

272views

### hashing a floating point number for test case purposes in the .Net runtime

I have complex test cases with geometric entities. It is easy for me to visually verify that the test case has passed by looking at the generated geometry in a viewport like so.
When I am satisfied ...

2votes

5answers

2kviews

### Join two strings into a single decimal value

I want to get a decimal value, for example 500.45, from two strings. How to achieve this operation?
decimal value; //like 500.45
string txt1, txt2;
txt1 = "500";
txt2 = "45";
value = Convert....

1vote

1answer

74views

### Strange double subtraction result in .NET

I'm aware of the inaccuracies of floating point operations but the results of the below code seem rather odd to be. Can anybody please explain it?
This is with .NET Framework 4.8 on Windows 10. The ...

1vote

0answers

126views

### Higher precision floating point calculations

I would like to find a way to make higher precision calculations in a compute shader using multiple floats to represent a single number.
My goal is a mandelbrot renderer where you can zoom deeper ...

1vote

0answers

46views

### Is it possible to manually increase the precision of a double as a bit string through calculations

In order to create a floating point number from a decimal value, we need to have a sign bit, and exponent, and a mantissa.
Double precision uses 11 exponent bits, and 52 mantissa bits.
Let's say we ...

1vote

0answers

74views

### Puzzled over Float vs Double benchmark

After searching SO for performance differences between floats and doubles in .NET, I put together a simple benchmark to test 2 methods, each with floats and doubles, debug and release. After running ...

1vote

0answers

83views

### How do I set the default decimal number separator in Python?

I am working on two applications, one running in C#/.NET and the other with the Python 3 interpreter targeting Windows 10 only. The two applications communicate with each other using a local network ...

1vote

0answers

63views

### Do .NET floating-point operations agree between x64 debug and release?

In this question, the asker wondered whether or not one could expect floating point operations to behave similarly between debug and release configurations.
The answers given there show that this is ...

1vote

0answers

246views

### using float.Parse() vs casting to get a value from double

I wasn't able to find a question that directly answers this question, I came across this line of code in a project at work (I added the comment)
// act.score is a double
float score = float.Parse(act....

1vote

2answers

148views

### Reversible decimal to double conversion

I'm curious to know if you could convert decimal to double, but still be able to come back to the same decimal I started with. Of course, assuming you use 15 or less significant digits. Basically, can ...

1vote

1answer

1kviews

### C# Decimal type rounding off inappropriately

I've got an in house banking solution for a summer camp that I'm a part of. The issue is coming up that some accounts are exactly a penny off at the end of the season.
I'm using SQL Server Express, ...

1vote

0answers

69views

### Formatting numbers in vertically aligned textboxes in C# with the floating point also vertically aligned

On my UI, there are several textboxes that are vertically aligned. I'd like to show the numbers in them in a way that all floating points are also vertically aligned. I also need to show the sign of ...

1vote

1answer

179views

### Could Mono.Simd or Microsoft.Bcl.Simd provide deterministic floating point arithmetic to C#?

C# cannot be coerced into deterministic floating point arithmetic, the designers seem to have considered the performance overhead too significant.
I'm trying to assess whether it would be possible to ...

1vote

0answers

557views

### Converting 4-byte OLE (VB6) to double/float in C#

Well, I have a 4 (four) bytes array that I need to convert to double data-type (or float) in C#.
I believe that the 4 bytes array is an OLE Automation one.
Tried the following, but no success:
// ...

1vote

1answer

457views

### Extjs float value is being passed as null in c# controller

I have a float value in extjs model. For some reason any value I enter and pass to c# controller, its value is always null. I checked the json data being pass and I can see the passed value.
This is ...

1vote

0answers

147views

### Byte[] to float conversion issue

I have a binary file written in big endian enviorment using c code.
er
On my PC a C# application is reading bytes from the file using BinaryReader and converting it to Managed struct using Marshalling....

1vote

2answers

578views

### Floats changing when passing them from CLI to C#

EDIT : i wrote the same code that i had at work at home to recreate the bug. this is the exact code !!
I'm trying to pass floating number from c++ to c# and i'm using c++ CLI as intermediate.
I have ...

1vote

2answers

483views

### C#: Store number in database with 2 places of decimal. Eliminate unwanted numbers

I have a field NumberValue1 declared like this
public double NumberValue1 { get; set; }
NumberValue1 has a datatype of Number in the
oracle database
I read in a value from an excel file which is
22....

1vote

0answers

2kviews

### Getting significand and exponent value from floating point literal with C#

A literal of 0.25 can be expressed with 1*2^(-2) so that we can get 1 as significand and -2 as exponent.
What function can I get the significand/exponent with C#?
int significand = GetSignificand(0....

1vote

2answers

47views

### Convert Long to Float Changing the number c#

I have long vairable long x = 231021578;
and when I convert it to float like float y = x;
the value of y will be 231021584
I want to know why this happen. Float is stored in 4 bytes and its range is ...

1vote

2answers

2kviews

### Error when trying to save data to a database with dbcontext. Not a valid instance of type float

I cant seem to figure this out. The error I get is
"The incoming tabular data stream (TDS) remote procedure call (RPC) protocol stream is >incorrect. Parameter 13 ("@10"): The supplied value is ...

0votes

0answers

55views

### Need a .NET Framework solution for IEEE-754 float/double parse and ToString

.NET framework does not give me expected results for float/double parsing and ToString. It is fixed in .NET 6 (even in .NET Core 3.0). However, I cannot migrate my whole project from .NET framework 4....

0votes

0answers

61views

### .NET 6.0 vs .NET Framework Number Rounding

As I was migrating a complex .NET Framework project to .NET 6.0, I noticed different results under some circumstances. I was able to trace down the issue to a basic inequality involving floats and a ...

0votes

0answers

33views

### I am having trouble evaluating the height in c#, when i introduce a number smaller than 1.5 still says come in

using System;
namespace Tarea_C_
{
class Program
{
static void Main(string[] args)
{
Console.Write("Write your age: ");
int age = Convert....

0votes

0answers

58views

### Is there a way to tell the compiler to compile with "fast-math" or something similar in C#?

In some other languages, you can allow the compiler to perform some floating point optimizations, one of which is rewriting your code into algebraically equivalent expressions. Here are some examples ...

0votes

0answers

54views

### How to insert value as 'null' instead of 0 to numeric column in sql using c#?

I need to insert float value as 'NULL' for all columns in sql instead of 0.00000 or 0.I have written code as below.Is there any better alternative for this since I should do this for all columns ...

0votes

0answers

63views

### C# - Converting data from Decimal to float is truncating after decimal points while retrieving data from database

I am trying to fetch percentage value from database and my database query gives 2 digits after decimal, but when i am trying to access that value in c# code, the decimal point and the 2 digits after ...

0votes

0answers

132views

### Why does float.TryParse() accepts "1.000e-45" and rounds it to 1.401298E-45?

I try to validate that an input string is a float. To achieve this I pass the input string like "1.000e-45" which is closer to 0 than the c# float minimum allows (1.401298E-45). I expect ...

0votes

1answer

36views

### Cutting corners with float-to-string representation?

I have a float I need to turn into a string with 5 decimals precision (X.XXXXX), which means I need to have at least 6 decimals for round up/down. The issue is that the operation to get integer ...

0votes

0answers

80views

### Why .net C# float is truncated to thousand in string representation?

How come the following is possible ?
immediate console:
"{0:F20},{1:F20}, {2:F20}".format(
res_ts, // float
AppMgr.logon_info.res_ts, //float
res_ts - AppMgr.logon_info.res_ts
)
"{0},{...

0votes

0answers

106views

### How to find the similarity (as a percentage) between two float[] arrays in c#

I need to get the similarity between two float[] arrays of the same lengths and return have this similarity be returned as a float which will be the percentage. How would you do this in c#?

0votes

1answer

485views

### Is it valid check for infinity with the equals operator?

c# how to check if a number is infinity
Answer: Double.IsInfinity, Double.IsPosiviteInfinity, etc.
So now comes my question: Is it valid to simply do:
double d = Double.PositiveInfinity;
if(d == ...

0votes

0answers

58views

### How to prevent Floating Point Excel Data loaded with SQL to be truncated

When I load floating point data from Excel in my C# software using SQL the data ends up being truncated.
For example, instead of 0,054034 I end up with 0,054.
The relevant part of my code is:
...

0votes

1answer

82views

### More elegant way to determine whether two floats have same signs in C#?

Is there any way that can compare the signs of the two floats quickly in C#? I doubt the performance of comparing a float with 0 again and again. However, it seems that bit operation of float is ...

0votes

0answers

58views

### Converting floats to ints reliably

I have an application that uses floats to handle distances, for example, 2.5f is 2.5 meters (250 centimeters). Now I want to interface with that application without using floats via network. I can't ...

0votes

0answers

84views

### Taking a Floating point number to 30th decimal point in C#

I have a number which is 0.010000000000000000000000000001. I want this number to be stored as it is and want to increment to it like 0.010000000000000000000000000002
0.010000000000000000000000000003
0....

0votes

1answer

56views

### IEEE floating point addition

instead of
float a = 32.342 , b = 193.132
float total = a + b
How can I convert them to 32 bits and add using their 32 bits explicitly?

0votes

0answers

258views

### Float Precision in the Visual Studio Immediate Window

Undertaking a code review I have come across some strange behaviour I don't understand in the C# immediate window.
all the variables below are floats and calculate the gradient of a line. ...