All Questions
Tagged with floating-point java
83
questions with no upvoted or accepted answers
3votes
1answer
62views
Dealing with floating point point inaccuracy in very small numbers efficiently
The program I am working with takes OpenStreetMap data to render a map. The data consists of 4 coordinates, that make up the bounds of the data.
I am drawing lines, that sometimes exceed these bounds ...
- 43
3votes
1answer
5kviews
Prevent Jackson from serializing a float as a double
Jackson seems to be coercing all floats into doubles in any data structure that I am trying to serialize into JSON. Is there any way to avoid this behavior?
Float f = 50.1f;
System.out.println(f); ...
- 334
2votes
0answers
212views
How can I get accurate results from math in scriptengine for java?
So I'm working on a big Java project and in it I have a part with user submitted equations, both mathematical and for things like If/else. I've been using a scriptengine with javascript to get the ...
2votes
0answers
175views
How can I find out if my JVM is using hardware square root?
If you read the top of the incredibly-hard-to-find native sqrt method for Java, which is located at jdk1.6\src\jdk\src\share\native\java\lang\fdlibm\src\e_sqrt.c you will find this:
/* __ieee754_sqrt(...
- 10.6k
2votes
2answers
423views
In Java currency representation, what are the pros and cons of using a long/short mantissa/exponent pattern over a double/epsilon pattern
Let me begin this question by stating that for the type of high performance application we are developing, BigDecimal is unacceptably slow. This cannot be compromised on.
In our domain, we will be ...
- 384
2votes
1answer
1kviews
DecimalFormat WITHOUT scientific notation
I have a DecimalFormat like this:
DecimalFormat df = new DecimalFormat("#,###.###");
Then I have 3 methods which just return a float value for time, potential and current, with many decimals in case ...
- 12.1k
1vote
1answer
51views
Is Java floating arithmetic division always within 1 ULP of the true result?
Is Java's floating arithmetic division always within 1 ULP of the true result? I read that CPUs sometimes do floating-point division a/b by doing a * 1/b. However, 1/b may be off by 1 ULP, and ...
- 502
1vote
3answers
357views
Determine if a variable is a float or an integer in java
I need to write a program which uses multiple methods called average which return the average of those values, if both are ints the value returned should be an int, and if one or more is a floating ...
- 21
1vote
1answer
160views
Mathematically correct result from Math.sqrt for int arguments?
Given an int x, is (int)Math.sqrt(x) guaranteed to return the mathematically correct result, truncated to an int (i.e., the exact mathematical value of floor(sqrt(x))) for all non-negative x?
- 55.9k
1vote
2answers
125views
Float.valueOf(String arg) rounding off issue in Java
People,
I stumbled upon a basic issue in my application where am trying to convert a string value to it's corresponding Float value :
Float val = obj.getFloat("220293.43") ;
System.out.println(val);
...
- 113
1vote
0answers
70views
How can a float value be trimmed to the first three digits?
its my first question here and maybe i'm just blind or extremely stupid, but i haven't found a matching answer to my problem in the past three days. Currently i'm coding an Idlegame (just to learn ...
- 11
1vote
1answer
106views
How do I round a float but keep it a float (i.e. without converting it to int)?
for example i have a result of circle area = 4.3, my question is 'when i get number grater than .5 [for example(4.6, 4.7, 4.8, 4.9)], i want it to give me 5.0, but if less than .5 [4.4, 4.3, 4.2, 4.1] ...
1vote
0answers
317views
Convert 80-bit Double to BigDecimal
tl;dr - How do I convert the binary of a IEEE 754 80-bit floating point number into a Java BigDecimal or Double?
I'm writing an .aiff file parser in Java following the specification found here.
I've ...
- 43
1vote
0answers
452views
Java writing output AudioInputStream + AudioFormat set wFormat to FLOAT?
I've confirmed my program "works" with a variety of signed integer output formats. However, when I attempt to use 32-bit floating point, the output audio metadata indicates it to be 32-bit signed ...
- 3,232
1vote
1answer
86views
How to know if a fraction will be rounded up when represented in floating point format (re: java remainder [%] results when using fp's)
Is there a simple way to tell whether a particular number gets rounded up in it's floating point representation? The reason I ask is related to a question I asked here and a similar question was asked ...
- 50
1vote
1answer
106views
Move an ImageView in a circular path
I need to move an ImageView in a circular path.
The Specifications for the program:
1) Each ImageView has a meteor class that contains the ImageView's
Current coordinates
Target coordinates
and a ...
user3458225
1vote
1answer
2kviews
MathContext.DECIMAL32 vs MathContext.DECIMAL64, which one to use, and why?
Should I use MathContext.DECIMAL32 or MathContext.DECIMAL64? I have looked at the documentation, but I couldn't really understand when to use either.
I'm using BigDecimal to represent a percentage ...
- 327
1vote
1answer
37views
When "extracting" float value from object of type Object is it better to use cast or .toString method?
I have an object containing float value. I need to "extract" it. Is it better to use cast float cellValue = (float) b; or .toString() method String cellValue = b.toString();? By better, I mean ...
- 159
1vote
0answers
626views
Adding two IEEE floating point numbers in Java
I am having a difficult time figuring out how to correctly add two IEEE floating point numbers using Java. I'm not sure how to proceed in actually adding the mantissas together because I don't get how ...
1vote
2answers
254views
Function to truncate values of resultset in java
I have to truncate the values obtained from database upto two fractional parts and put that values in the resultset I'm adding values from resultset into list in following way
while(rs.next())
...
- 526
1vote
0answers
2kviews
Converting from Double to Binary
I've been a lot of trouble figuring this class problem. My due date is tomorrow and I still don't know how to do it. I made a code when the input put by the user is converted into binary, octal, and ...
1vote
1answer
1kviews
Why I insert double/float column into Cassandra by hector and got incorrect value int database
I have Some question about Hector Insert double/float data into Cassandra
new Double("13.45")------->13.468259733915328
new Float("64.13") ------->119.87449
When I insert data into Cassandra ...
- 11
1vote
1answer
184views
safeguard against floating point errors in java
As the number of frames rendered in a game increase beyond a certain
limit, instead of obtaining an identity matrix for the following type of matrix transformation:
Matrix.setIdentity(ModelMatrix);
...
- 31
1vote
0answers
343views
Java Formatter, float values, exception on exceeding maximum width
I want to use the java.util.Formatter class for formatting fixed-length records. The record includes Integer and Double fields.
For any Integer and Double for which the corresponding string ...
- 787
1vote
1answer
1kviews
Crash on scene load - java.lang.ClassCastException
I believe the following error to be some variable somewhere in my many files to be a float used like a int or somthing like that:
java.lang.ClassCastException: java.lang.Float
But for the life of me,...
- 11
1vote
3answers
798views
Processing - when parsing string to float NullPointerException is thrown
I have a string -0.06,0.00,-0.99. I want to parse the first number to a float.
I am using:
String myString = "-0.06,0.00,-0.99";
float myFloat = float(split(myString, ",")[0]);
println(split(...
- 584
0votes
0answers
56views
as.character() with numeric loses precision when a string is transformed into numeric
In R I want to run following code:
str <- "2811.1019192874087"
val <- as.numeric(str)
str2 <- as.character(val)
print(str2)
print(val)
It prints:
"2811.10191928741"
2811....
- 1,526
0votes
0answers
26views
Java's Double.doubleToLongBits equivalent in MySQL
Suppose number: -58.725. When calling Double.doubleToLongBits(-58.725) it returns: -4589910410541609779. How can i get the same result in MySQL?
I have tried: CONV(CONV(-58.725, 10, 2), 2, 10) which ...
- 192
0votes
0answers
13views
How to increase Gephi allowed digit precision?
Is it possible for Gephi (https://gephi.org/) to read .graphml files containing edges with very high digit precision weight values ?
For example numbers like: 1.0E-180 ?
By default Gephi treats ...
- 67
0votes
1answer
147views
How to get the exact value in BigDecimal?
I want to get a BigDecimal value with the following attributes:
Rounding mode: Halfe-even
number of digits after the point: 2
I have the following code:
public BigDecimal standardDeviation() {
...
0votes
0answers
29views
How to convert .wav to float in Android Studio?
I am working on a machine learning project with tensorflow on android, and I need to convert .wav files to float arrays in order to do some processing on them. In Python I used soundfile library with ...
- 11
0votes
0answers
55views
PostgresSQL Driver (version 12) rounding Java float Value to six digits
I am using postgresql version 9.4.1.
I am in the process of upgrading the postgres driver from 9.1-901.jdbc4 (maven: postgresql:postgresql) to 42.2.5 (maven: org.postgresql: postgresql)
While doing so,...
- 443
0votes
0answers
84views
Arbitrary Precision Data Types In Java
Suppose my architecture supports IEEE 754. This means I'll have 24b for single precision and 53b for double precision. Now supposing in my program, I want to use arbitrary precision - suppose I want ...
- 3,604
0votes
1answer
245views
What is the MessageFormat pattern to produce localized, full precision output for any double
Given the example code:
String format = "{0,number}"; // <- What do I put here?
double value = Math.PI * 1e-10;
System.out.println(value);
System.out.println(new MessageFormat(format, Locale.US)....
- 478
0votes
1answer
299views
What are the rules governing Narrowing conversion from double to float?
Is there a magnitude (decimal range) on which the conversion from double to float is based ?
At the beginning I was thinking so but the exponent part confused me ..
So what rules are governing that ...
0votes
0answers
73views
What is the point of using custom defined epsilon to avoid errors from operations on floating point numbers?
While solving the following problem from CTCI book:
Given a two-dimensional graph with points on it, find a line which passes the most number of points.
I came across this solution. It essentially ...
- 1,272
0votes
0answers
142views
JPQL number to string conversion disagreeing with Java
Due to some of the restrictions of JPQL (no ordering in subqueries, can't check array equality), I'm having to do some workarounds. Namely, I'm concatenating some numbers (and commas) into a string ...
- 3,891
0votes
0answers
129views
Java Byte to Float
I reveive 2 Byte and I want to represent them as float.
I know about the Method
float f = ByteBuffer.wrap(weight).order(ByteOrder.LITTLE_ENDIAN).getFloat();
But this Method only worky with 4 byte ...
- 1,430
0votes
0answers
332views
How can I check if an array value in an array of floats is null/does not contain a value?
I need to go through an array of 2D float values, checking if the array has a value or is null. I am confused on how I can check if the array value is null, since it's an array of floats?
- 81
0votes
0answers
771views
How can I get 9 digits (to left of decimal point) precision for float (wikipedia promises [6-9] digits, but math allows only 7?
Double-precision (IEEE 754): 53-bit significand precision gives from 15 to 17 significant decimal digits precision (wikipedia).
Single-precision (IEEE 754): This gives from 6 to 9 significant decimal ...
- 2,834
0votes
1answer
565views
Efficient way to encode float to a UTF-8 byte buffer in java
I want to encode float to UTF-8 byte array in java. Are there any libraries or packages that do this in an efficient way.
- 87
0votes
0answers
82views
Simplest approach to make a calculator?
I've read other questions on this and they're not quite as general as I want this discussion to be.
My question is as stated: What's the simplest approach to making a basic calculator, say with ...
- 310
0votes
0answers
804views
Why SQL float maps to Java double instead of Java float?
As part of my graduation thesis, I'm looking up how SQL standard types (not implementation-specific types) map to Java types. Most of it are no-brainers but I noticed one thing: SQL float is listed ...
- 105
0votes
0answers
81views
How to write floating point intensity into images? And then to read it back as floating point again?
I'm currently working with Java, so if there's any function that I can work with Java itself then it'll be great.(Any other language is also fine as long as it works) I cannot convert the floating ...
0votes
0answers
254views
JFormattedTextField.getValue is returning not formatted value
I would like to have JFormattedTextField to only accept numbers, that is a double number with a maximum of 2 fraction digits. I read the information present on JFormattedTextField tutorial on Oracle ...
- 122
0votes
2answers
90views
Why zero absent when convert float numbers (which are between 0 and 1) to string
In Oracle database, I store a number 0.5 in a column with format number(17, 2).
And I read it using a Java program (with mybatis), and store as a string field of an entity object. The value of this ...
- 73
0votes
1answer
76views
logic error? diving 2 int that result in a float
I am writing a combat simulator. My attacker class initiates an attack and my defender class is supposed to block it. manager is my main class that computes and prints the results. My problem is with ...
- 21
0votes
0answers
1kviews
when reading float numbers from excel by apache poi in java, float numbers considered as type of 'CELL_TYPE_STRING'
I need to read only numeric data from excel file in java. I read all data by this code successfully:
try
{
FileInputStream file = new FileInputStream(new File("book1.xlsx"));
//Create ...
- 115
0votes
1answer
872views
Data truncated for column when using FLOAT in GUI
I am using a video course on database programming with the current lesson being using Java to connect to MySQL. I have followed the video, and even copied the text working file for this particular ...
- 5
0votes
1answer
948views
GWT: Get current time as a float since long isn't supported
I am making a GWT game using LibGDX and it relies on using the current time to compare differences between the server state and the client state.
I am currently using longs that I get using System....
- 970