# All Questions

Tagged with floating-point type-conversion

39
questions with no upvoted or accepted answers

9votes

1answer

596views

### Are .NET Decimal type computations deterministic?

I have two questions regarding .NET's decimal data type determinism:
Are decimal type computations cross-platform deterministic? Or in other words, will math operations on decimal type produce ...

2votes

0answers

47views

### Understanding IEEE 754: why convertFromInt and convertToIntegerXXX are categorized as arithmetic operations and not conversion operations?

Note: understanding IEEE 754. Please be patient.
IEEE Std 754-2019:
5.4.1 Arithmetic operations
― formatOf-convertFromInt(int)
― intFormatOf-convertToIntegerXXX(source)
Question: why convertFromInt ...

2votes

1answer

568views

### Casting from (long)double to size_t

I'm trying to implement Sieve of Eratosthenes as efficient as possible. I'd like to set length of my prime array to upper bond of
pi(n) < 1.25506n / ln n
but I'm not sure how to proceed with ...

2votes

2answers

2kviews

### Rcpp: Converting SEXP to float/double

SEXP callFunction1(List network, List words, Function testWordContinuity){
SEXP res = testWordContinuity(network, words);
return res;
}
for(int i=0; i<(n_epochs); i++){
NumericVector ...

2votes

0answers

569views

### Facing undefined symbols linker issue with Diab compiler when I type cast array of data from float to long long

I wrote a small example code and executed in both GCC and DIAB compilers.
#include<stdio.h>
int main()
{
float a[10];
long long int b[10];
int i;
for (i =0;i<10;i++)
{
...

2votes

3answers

1kviews

### Inconsistent type conversion in python/numpy when using scalars or lists/arrays

I have a question about the strange way python/numpy performs type conversion. When I perform an arithmetic operation between a float32 and a float64 number, the lower precision is converted to ...

2votes

1answer

2kviews

### SSE: convert from const __m128 * to const float *

I'm trying to write a little SSE code but can't continue because of this error:
error C2664: '_mm_loadu_ps' : cannot convert parameter 1 from 'const __m128 *' to 'const float *'
I've to load ...

1vote

1answer

113views

### floor-then-convert_int vs convert_int_rtn - equivalent in OpenCL?

In OpenCL, is:
convert_int(floor(my_float))
exactly equivalent to
convert_int_rtn(my_float)
semantically? Would I ever prefer the former command to the latter?
If you answer depends on the fact ...

1vote

2answers

125views

### Float.valueOf(String arg) rounding off issue in Java

People,
I stumbled upon a basic issue in my application where am trying to convert a string value to it's corresponding Float value :
Float val = obj.getFloat("220293.43") ;
System.out.println(val);
...

1vote

0answers

1kviews

### C programming: creating a function to convert double to string

I am trying to code a function that converts a double into a string (a sort of dtoa function). I don't want to use any of the standard library that will do all the job for me (itoa is ok, strlen too, ...

1vote

1answer

614views

### Conversion of hex to Float in SciLab

I'm trying to read a float number in a hex representation f.ex
43079A11
which should give me 135.601822, (similar to what this site does http://babbage.cs.qc.cuny.edu/IEEE-754.old/32bit.html)
using ...

1vote

0answers

1kviews

### mysql float data type round off's values after 6 decimal places

I have a table (say table1) with one field of data type float (say field1). The values in this field are all unique with upto 8 decimal places.
I executed following queries on this table:
SELECT ...

1vote

2answers

47views

### Convert Long to Float Changing the number c#

I have long vairable long x = 231021578;
and when I convert it to float like float y = x;
the value of y will be 231021584
I want to know why this happen. Float is stored in 4 bytes and its range is ...

0votes

0answers

575views

### ValueError: could not convert string to float using Pandas

Here I would like to convert all column values to floating point numbers, I have tried the code below but getting error, I have tried and removed ',' (as you can see in comments) but still getting ...

0votes

3answers

60views

### How can i round a fraction properly to two places after the decimal?

I've written a simple program, which a user types an amount of money, and the computer returns some bills to change. When I type 99.99, the computer cannot handle properly with 0.01 fraction. roundf() ...

0votes

0answers

24views

### idx = int(float(img_id)) ValueError: could not convert string to float:

I am getting this error when I try to read my files from a given directory. My code is:
box_dir = '/home/test/'
img_id = img_metas[0]['filename'][34:40]
idx = int(float(img_id))
...

0votes

0answers

54views

### How to insert value as 'null' instead of 0 to numeric column in sql using c#?

I need to insert float value as 'NULL' for all columns in sql instead of 0.00000 or 0.I have written code as below.Is there any better alternative for this since I should do this for all columns ...

0votes

2answers

346views

### STM32L1 snprintf usage to convert float to string

I have problem with convert float to str on STM32,
I have two simple functions to print Int and Float
Int work fine, but float print cause program crash
void LCD_PrintInt(int value) {
char str[8];
...

0votes

1answer

37views

### Need help converting string or int to float

So I have been stuck on this one for a while and could use some help. I have been trying to fix this code and I keep getting an error about invalid syntax. So here is the code I need help with need to ...

0votes

0answers

352views

### Data type conversion in GNURadio

I'm reading a stream of float digits from an external program to my GNURadio flowchart through "Socket PDU" bloc. However, the latter delivers a stream of strings instead of floats. I tried writting ...

0votes

1answer

299views

### What are the rules governing Narrowing conversion from double to float?

Is there a magnitude (decimal range) on which the conversion from double to float is based ?
At the beginning I was thinking so but the exponent part confused me ..
So what rules are governing that ...

0votes

0answers

142views

### JPQL number to string conversion disagreeing with Java

Due to some of the restrictions of JPQL (no ordering in subqueries, can't check array equality), I'm having to do some workarounds. Namely, I'm concatenating some numbers (and commas) into a string ...

0votes

0answers

35views

### Packing integer as floats, finding original ints

We are packing two 12 bits integers into three 8 bits RGB colors to convey values to a GPU.
The process goes like this:
Two 16 bits integers are generated (U and V)
They are decimated to two 12 ...

0votes

0answers

73views

### Matlab structure with int32 and float values

I created a function that generates a structure to store parameters acquired during a simulation. Some parameters are int32, while other (actually, most of them) are float. The structure is generated ...

0votes

0answers

31views

### Type Conversion for data transmission

i have a problem converting data between different types. I know there are a lot of Questions and How-to's regarding this topic, but i wasn't able to succeed with any of those.
I have a transmission ...

0votes

3answers

906views

### Objective-C convert 2 byte into a float

I have 2 byte temperature value like:
[0x0C, 0xDE] and I need to convert it to a float, what I've tried to do is:
NSData *payload <-- this contains the bytes in the "range" defined after
[payload ...

0votes

2answers

337views

### Why cant we typecast Float(wrapper) to int(primitive) in java?

While I'm doing typecast from float to int type and typecast from Integer to int it is working. But while i am trying to typecast from Float to int type I'm getting the error as "incompatible type". ...

0votes

1answer

136views

### Something goes wrong when I convert NSString to float

Something goes wrong when I convert NSString to float. I got zero value, but why?
NSString *polylineStr = @"51.351305,-0.024239;51.374855,-0.092412";
NSArray *strPolylineArr = [polylineStr ...

0votes

2answers

2kviews

### converting float to fixed point in python

I think I am overthinking this probelem, but I can't seem to solve it. I am reading in little endian floats from a file (range (-1,1)), and want to convert them to fixed point 32b.
If I read in the ...

0votes

1answer

222views

### How do I convert a double to a proprietary floating point format in C#?

I have a 20 bit proprietary floating point format defined as:
Bits: 19 18 17 16 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00
Use: s e e e e e m m m m m m m m m m m m m m
...

0votes

1answer

1kviews

### Smallest positive number with the following format

12 bit floating point number: 1 bit sign, 4 bit exponent, 7 bit significand.
The significand is normalized so that the first digit to the right of the radix point must be a 1.
Exponent is signed 2's ...

0votes

0answers

133views

### fixed point datatype with PHP 5

Am I wrong (I hope), but I don't find on PHP a fixed-point datatype !?
Datatype like DECIMAL/NUMERIC from MySQL will be nice for doing operations on money without precision loss.
Any ideas ?

0votes

1answer

2kviews

### inserting c# double values to sql server float field. (no fractional part)

I am trying to insert double values to my database via EF 5. I generated EF entity model from db. There is a price column in the table which is float, and naturally EF generated a double type for the ...

0votes

1answer

4kviews

### Converting a float to an integer

There is a catch!
I have a IEEE 754 single precision (32 bit) float stored in two consecutive 16bit integers.
The processor I am using doesn't have floating point maths or float data types! What I ...

-1votes

1answer

104views

### converting doubles to long doubles, understanding of 'binary vs. decimal' ambiguity? and is there a standard solution?

[edit 2021-09-26]
sorry!, I have to admit that I asked crap here, explanation follows. I don't think I should post this as an 'answer', so as an edit:
I'm still curious how a 'double' value of 0.1 ...

-1votes

1answer

212views

### float.Parse(string) without exponent in C#

Pretty simple situation, but I got stuck exactly here. So, I have next method:
public static float? ToSingleNullable(string value)
{
if (string.IsNullOrEmpty(value))
return null;
else
...

-1votes

1answer

3kviews

### 64 Bit Hex conversion to decimal

I have a 16 Bit hex value:
4164ACFCE33404EA
After conversion, the number is:
10840039.1001
I've tried both IEEE single and double precision 64-bit to decimal and a variety of 16 bit two ...

-1votes

1answer

164views

### Convert a float to an int

I have a price for a car, let's say 10000. I want to apply a 20% sale to this price.
I have a struct in which auta->cena is a float.
int year, i, n=0, pocet=1, sale;
scanf(" %d", &year);
...

-2votes

2answers

164views

### How to check the input format in java

i need to check whether the number input is in decimal format or in floating point format in java coding.
in simple terms how would this check be possible?