All Questions

Tagged with
83 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3votes
1answer
62views

Dealing with floating point point inaccuracy in very small numbers efficiently

The program I am working with takes OpenStreetMap data to render a map. The data consists of 4 coordinates, that make up the bounds of the data. I am drawing lines, that sometimes exceed these bounds ...
user avatar
3votes
1answer
5kviews

Prevent Jackson from serializing a float as a double

Jackson seems to be coercing all floats into doubles in any data structure that I am trying to serialize into JSON. Is there any way to avoid this behavior? Float f = 50.1f; System.out.println(f); ...
user avatar
2votes
0answers
212views

How can I get accurate results from math in scriptengine for java?

So I'm working on a big Java project and in it I have a part with user submitted equations, both mathematical and for things like If/else. I've been using a scriptengine with javascript to get the ...
user avatar
2votes
0answers
175views

How can I find out if my JVM is using hardware square root?

If you read the top of the incredibly-hard-to-find native sqrt method for Java, which is located at jdk1.6\src\jdk\src\share\native\java\lang\fdlibm\src\e_sqrt.c you will find this: /* __ieee754_sqrt(...
user avatar
  • 10.6k
2votes
2answers
423views

In Java currency representation, what are the pros and cons of using a long/short mantissa/exponent pattern over a double/epsilon pattern

Let me begin this question by stating that for the type of high performance application we are developing, BigDecimal is unacceptably slow. This cannot be compromised on. In our domain, we will be ...
user avatar
  • 384
2votes
1answer
1kviews

DecimalFormat WITHOUT scientific notation

I have a DecimalFormat like this: DecimalFormat df = new DecimalFormat("#,###.###"); Then I have 3 methods which just return a float value for time, potential and current, with many decimals in case ...
user avatar
  • 12.1k
1vote
1answer
51views

Is Java floating arithmetic division always within 1 ULP of the true result?

Is Java's floating arithmetic division always within 1 ULP of the true result? I read that CPUs sometimes do floating-point division a/b by doing a * 1/b. However, 1/b may be off by 1 ULP, and ...
user avatar
1vote
3answers
357views

Determine if a variable is a float or an integer in java

I need to write a program which uses multiple methods called average which return the average of those values, if both are ints the value returned should be an int, and if one or more is a floating ...
user avatar
1vote
1answer
160views

Mathematically correct result from Math.sqrt for int arguments?

Given an int x, is (int)Math.sqrt(x) guaranteed to return the mathematically correct result, truncated to an int (i.e., the exact mathematical value of floor(sqrt(x))) for all non-negative x?
user avatar
  • 55.9k
1vote
2answers
125views

Float.valueOf(String arg) rounding off issue in Java

People, I stumbled upon a basic issue in my application where am trying to convert a string value to it's corresponding Float value : Float val = obj.getFloat("220293.43") ; System.out.println(val); ...
user avatar
1vote
0answers
70views

How can a float value be trimmed to the first three digits?

its my first question here and maybe i'm just blind or extremely stupid, but i haven't found a matching answer to my problem in the past three days. Currently i'm coding an Idlegame (just to learn ...
user avatar
  • 11
1vote
1answer
106views

How do I round a float but keep it a float (i.e. without converting it to int)?

for example i have a result of circle area = 4.3, my question is 'when i get number grater than .5 [for example(4.6, 4.7, 4.8, 4.9)], i want it to give me 5.0, but if less than .5 [4.4, 4.3, 4.2, 4.1] ...
user avatar
1vote
0answers
317views

Convert 80-bit Double to BigDecimal

tl;dr - How do I convert the binary of a IEEE 754 80-bit floating point number into a Java BigDecimal or Double? I'm writing an .aiff file parser in Java following the specification found here. I've ...
user avatar
1vote
0answers
452views

Java writing output AudioInputStream + AudioFormat set wFormat to FLOAT?

I've confirmed my program "works" with a variety of signed integer output formats. However, when I attempt to use 32-bit floating point, the output audio metadata indicates it to be 32-bit signed ...
user avatar
  • 3,232
1vote
1answer
86views

How to know if a fraction will be rounded up when represented in floating point format (re: java remainder [%] results when using fp's)

Is there a simple way to tell whether a particular number gets rounded up in it's floating point representation? The reason I ask is related to a question I asked here and a similar question was asked ...
user avatar
1vote
1answer
106views

Move an ImageView in a circular path

I need to move an ImageView in a circular path. The Specifications for the program: 1) Each ImageView has a meteor class that contains the ImageView's Current coordinates Target coordinates and a ...
user avatar
1vote
1answer
2kviews

MathContext.DECIMAL32 vs MathContext.DECIMAL64, which one to use, and why?

Should I use MathContext.DECIMAL32 or MathContext.DECIMAL64? I have looked at the documentation, but I couldn't really understand when to use either. I'm using BigDecimal to represent a percentage ...
user avatar
  • 327
1vote
1answer
37views

When "extracting" float value from object of type Object is it better to use cast or .toString method?

I have an object containing float value. I need to "extract" it. Is it better to use cast float cellValue = (float) b; or .toString() method String cellValue = b.toString();? By better, I mean ...
user avatar
  • 159
1vote
0answers
626views

Adding two IEEE floating point numbers in Java

I am having a difficult time figuring out how to correctly add two IEEE floating point numbers using Java. I'm not sure how to proceed in actually adding the mantissas together because I don't get how ...
user avatar
1vote
2answers
254views

Function to truncate values of resultset in java

I have to truncate the values obtained from database upto two fractional parts and put that values in the resultset I'm adding values from resultset into list in following way while(rs.next()) ...
user avatar
1vote
0answers
2kviews

Converting from Double to Binary

I've been a lot of trouble figuring this class problem. My due date is tomorrow and I still don't know how to do it. I made a code when the input put by the user is converted into binary, octal, and ...
user avatar
1vote
1answer
1kviews

Why I insert double/float column into Cassandra by hector and got incorrect value int database

I have Some question about Hector Insert double/float data into Cassandra new Double("13.45")------->13.468259733915328 new Float("64.13") ------->119.87449 When I insert data into Cassandra ...
user avatar
1vote
1answer
184views

safeguard against floating point errors in java

As the number of frames rendered in a game increase beyond a certain limit, instead of obtaining an identity matrix for the following type of matrix transformation: Matrix.setIdentity(ModelMatrix); ...
user avatar
1vote
0answers
343views

Java Formatter, float values, exception on exceeding maximum width

I want to use the java.util.Formatter class for formatting fixed-length records. The record includes Integer and Double fields. For any Integer and Double for which the corresponding string ...
user avatar
1vote
1answer
1kviews

Crash on scene load - java.lang.ClassCastException

I believe the following error to be some variable somewhere in my many files to be a float used like a int or somthing like that: java.lang.ClassCastException: java.lang.Float But for the life of me,...
user avatar
  • 11
1vote
3answers
798views

Processing - when parsing string to float NullPointerException is thrown

I have a string -0.06,0.00,-0.99. I want to parse the first number to a float. I am using: String myString = "-0.06,0.00,-0.99"; float myFloat = float(split(myString, ",")[0]); println(split(...
user avatar
0votes
0answers
56views

as.character() with numeric loses precision when a string is transformed into numeric

In R I want to run following code: str <- "2811.1019192874087" val <- as.numeric(str) str2 <- as.character(val) print(str2) print(val) It prints: "2811.10191928741" 2811....
user avatar
  • 1,526
0votes
0answers
26views

Java's Double.doubleToLongBits equivalent in MySQL

Suppose number: -58.725. When calling Double.doubleToLongBits(-58.725) it returns: -4589910410541609779. How can i get the same result in MySQL? I have tried: CONV(CONV(-58.725, 10, 2), 2, 10) which ...
user avatar
0votes
0answers
13views

How to increase Gephi allowed digit precision?

Is it possible for Gephi (https://gephi.org/) to read .graphml files containing edges with very high digit precision weight values ? For example numbers like: 1.0E-180 ? By default Gephi treats ...
user avatar
  • 67
0votes
1answer
147views

How to get the exact value in BigDecimal?

I want to get a BigDecimal value with the following attributes: Rounding mode: Halfe-even number of digits after the point: 2 I have the following code: public BigDecimal standardDeviation() { ...
user avatar
0votes
0answers
29views

How to convert .wav to float in Android Studio?

I am working on a machine learning project with tensorflow on android, and I need to convert .wav files to float arrays in order to do some processing on them. In Python I used soundfile library with ...
user avatar
  • 11
0votes
0answers
55views

PostgresSQL Driver (version 12) rounding Java float Value to six digits

I am using postgresql version 9.4.1. I am in the process of upgrading the postgres driver from 9.1-901.jdbc4 (maven: postgresql:postgresql) to 42.2.5 (maven: org.postgresql: postgresql) While doing so,...
user avatar
  • 443
0votes
0answers
84views

Arbitrary Precision Data Types In Java

Suppose my architecture supports IEEE 754. This means I'll have 24b for single precision and 53b for double precision. Now supposing in my program, I want to use arbitrary precision - suppose I want ...
user avatar
0votes
1answer
245views

What is the MessageFormat pattern to produce localized, full precision output for any double

Given the example code: String format = "{0,number}"; // <- What do I put here? double value = Math.PI * 1e-10; System.out.println(value); System.out.println(new MessageFormat(format, Locale.US)....
user avatar
  • 478
0votes
1answer
299views

What are the rules governing Narrowing conversion from double to float?

Is there a magnitude (decimal range) on which the conversion from double to float is based ? At the beginning I was thinking so but the exponent part confused me .. So what rules are governing that ...
user avatar
0votes
0answers
73views

What is the point of using custom defined epsilon to avoid errors from operations on floating point numbers?

While solving the following problem from CTCI book: Given a two-dimensional graph with points on it, find a line which passes the most number of points. I came across this solution. It essentially ...
user avatar
  • 1,272
0votes
0answers
142views

JPQL number to string conversion disagreeing with Java

Due to some of the restrictions of JPQL (no ordering in subqueries, can't check array equality), I'm having to do some workarounds. Namely, I'm concatenating some numbers (and commas) into a string ...
user avatar
  • 3,891
0votes
0answers
129views

Java Byte to Float

I reveive 2 Byte and I want to represent them as float. I know about the Method float f = ByteBuffer.wrap(weight).order(ByteOrder.LITTLE_ENDIAN).getFloat(); But this Method only worky with 4 byte ...
user avatar
0votes
0answers
332views

How can I check if an array value in an array of floats is null/does not contain a value?

I need to go through an array of 2D float values, checking if the array has a value or is null. I am confused on how I can check if the array value is null, since it's an array of floats?
user avatar
0votes
0answers
771views

How can I get 9 digits (to left of decimal point) precision for float (wikipedia promises [6-9] digits, but math allows only 7?

Double-precision (IEEE 754): 53-bit significand precision gives from 15 to 17 significant decimal digits precision (wikipedia). Single-precision (IEEE 754): This gives from 6 to 9 significant decimal ...
user avatar
0votes
1answer
565views

Efficient way to encode float to a UTF-8 byte buffer in java

I want to encode float to UTF-8 byte array in java. Are there any libraries or packages that do this in an efficient way.
user avatar
  • 87
0votes
0answers
82views

Simplest approach to make a calculator?

I've read other questions on this and they're not quite as general as I want this discussion to be. My question is as stated: What's the simplest approach to making a basic calculator, say with ...
user avatar
0votes
0answers
804views

Why SQL float maps to Java double instead of Java float?

As part of my graduation thesis, I'm looking up how SQL standard types (not implementation-specific types) map to Java types. Most of it are no-brainers but I noticed one thing: SQL float is listed ...
user avatar
0votes
0answers
81views

How to write floating point intensity into images? And then to read it back as floating point again?

I'm currently working with Java, so if there's any function that I can work with Java itself then it'll be great.(Any other language is also fine as long as it works) I cannot convert the floating ...
user avatar
0votes
0answers
254views

JFormattedTextField.getValue is returning not formatted value

I would like to have JFormattedTextField to only accept numbers, that is a double number with a maximum of 2 fraction digits. I read the information present on JFormattedTextField tutorial on Oracle ...
user avatar
  • 122
0votes
2answers
90views

Why zero absent when convert float numbers (which are between 0 and 1) to string

In Oracle database, I store a number 0.5 in a column with format number(17, 2). And I read it using a Java program (with mybatis), and store as a string field of an entity object. The value of this ...
user avatar
  • 73
0votes
1answer
76views

logic error? diving 2 int that result in a float

I am writing a combat simulator. My attacker class initiates an attack and my defender class is supposed to block it. manager is my main class that computes and prints the results. My problem is with ...
user avatar
0votes
0answers
1kviews

when reading float numbers from excel by apache poi in java, float numbers considered as type of 'CELL_TYPE_STRING'

I need to read only numeric data from excel file in java. I read all data by this code successfully: try { FileInputStream file = new FileInputStream(new File("book1.xlsx")); //Create ...
user avatar
  • 115
0votes
1answer
872views

Data truncated for column when using FLOAT in GUI

I am using a video course on database programming with the current lesson being using Java to connect to MySQL. I have followed the video, and even copied the text working file for this particular ...
user avatar
  • 5
0votes
1answer
948views

GWT: Get current time as a float since long isn't supported

I am making a GWT game using LibGDX and it relies on using the current time to compare differences between the server state and the client state. I am currently using longs that I get using System....
user avatar
  • 970

15 30 50 per page