All Questions

74 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3votes
1answer
431views

Lack of precision of the toFixed method in javascript

I have do some test about Number.prototype.toFixed method in chrome(v60.0.3112.101) console and found sth puzzled me. Why 1.15.toFixed(1) return "1.1" but not the "1.2"? Why 1.05.toFixed(1) return "1....
user avatar
  • 1,199
3votes
1answer
124views

Javascript wrongfully changes the result of a simple multiplication. How can I fix it?

function roundUp(num, precision) { return Math.ceil(num * precision) / precision; } var num = 0.07; var precision = 100; console.log(roundUp(num, precision)); When the arguments to the ...
user avatar
  • 1,154
3votes
0answers
115views

Behaviour of floating point precision for division

While working with various floating point number solutions I have been logging the values to compare the outputs. e.g. console.log(3 * 0.1) //0.30000000000000004 console.log(3 * 0.2) //0....
user avatar
  • 1,441
2votes
0answers
655views

Handle money and currencies in javascript multiplying and dividing by 100

I'm writing an app for myself. I need to handle money and currencies. I read this and I want to try the second approach: You can multiply floats into integers before you calculate, then divide them ...
user avatar
  • 1,325
2votes
1answer
4kviews

Parse float with decimals to JSON in JavaScript also for 1 as 1.00

The situation: My sensor measures data, that I process in a NodeRED-function and afterwards parse into a JSON-object. The NodeRED-function allows me to write JavaScript-Code. The JSON-object gets send ...
user avatar
2votes
1answer
99views

Floating Point Number issue JavaScript

I'm struggling to resolve a Floating Point Number issue where var change returns as 0.0999 recurring, and i need to return 0.01 (one penny). The code works fine, except the very last penny, because of ...
user avatar
2votes
0answers
212views

How can I get accurate results from math in scriptengine for java?

So I'm working on a big Java project and in it I have a part with user submitted equations, both mathematical and for things like If/else. I've been using a scriptengine with javascript to get the ...
user avatar
2votes
2answers
6kviews

JSON.parse fails for negative floating point numbers

I have a simple script like this: request = $.ajax({ url: "/getmesomefloats.php", type: "post", }); request.done(function (response, textStatus, jqXHR){ ...
user avatar
2votes
2answers
484views

How to not lose precision on division and multiplication?

I have operation like this: const amount = 100; const rate = 0.32123; const a = amount * rate; a // 32.123000000000005 const b = a / rate; b // 100.00000000000001 I need numbers to be the same ...
user avatar
2votes
1answer
228views

How to convert string value to Number for exact calculation ? Javascript

Below is the code : var a ="10.9549999999999141"; console.log(a); console.log(typeof a); console.log(Number(a)); Now the above code results as below : "10.9549999999999141" "string" 10....
user avatar
1vote
0answers
32views

toFixed()'s issues on pushing items into an array

In function giveChange I am returning a two-dimensional array with the 2 values being banknote and change. However, whenever toFixed() is added to limit the decimal places of a floating number, the ...
user avatar
1vote
2answers
67views

JavaScript: parseFloat() strips trailing zeros

I have been trying to find a solution to the following problem. I have a string that is a floating-point number like, var value = '12.30'; When I try to cast it to be a floating number parseFloat('...
user avatar
  • 1,824
1vote
1answer
103views

Json string number format

I try to understand JSON strings better. Especially numbers in that format. Checking if a JSON string is a valid number is easy (as described later), but what are the conventions for parsing a number ...
user avatar
1vote
1answer
263views

Object conversion with JavaScript UDF in Snowflake (floats)

Consider the following JavaScript UDF in Snowflake: create or replace function PUBLIC.TEST(F FLOAT) returns VARIANT language JAVASCRIPT returns NULL on NULL INPUT immutable comment ...
user avatar
1vote
1answer
68views

Determine precision of number, including trailing zeros

I suspect that this isn't possible, but giving it a try: Is it possible to determine the number of decimal points of a number, whether or not those decimal points are trailing zeros? getPrecision(3) ...
user avatar
  • 1,649
1vote
1answer
411views

Node Buffer.readFloatBE() extends data to 64-bit

When converting data to/from a Buffer using 32-bit floating point numbers, it seems that nodejs extends the float to double using some criteria that makes the transformation not reversible: > f=3....
user avatar
  • 673
1vote
0answers
76views

Converting a Twos-complement number to its binary representation

I am doing some bitwise manipulation. I am adding a 32 bit number to another 32 bit number with |= instead of += because I was thinking the number might be getting messed up when it converts to 32 ...
user avatar
1vote
1answer
598views

MongoDB number precision when using $inc in node/mongoose

I'm currently modeling a database using MongoDB in which users can transfer funds between accounts and buy products, the values of which are debited from their current balances. I'm working with a ...
user avatar
1vote
1answer
3kviews

Javascript: Building a cash register function and having trouble with floating points

I have the following code setup for a cash register function I'm building for freecodecamp. I noticed several issues with floating points so I took some other peoples advice and added .toFixed(2) ...
user avatar
  • 551
1vote
0answers
41views

Reduction of a JavaScript object with numeric keys and float values causes unpredictable behaviour on iOS

Consider the following two objects: const ReductionMap = { '10': 10.5, '1000': 20.5, } const initialValues = { '10': 110, '1000': 120, } Using lodash v4.17.5 (here _), we reduce ...
user avatar
1vote
1answer
321views

Is double arithmetic the same in cpp and javascript?

Context: I'm trying to port into c++ the implementation of a hash function that uses double arithmetic from its original implementation in JavaScript. In this function, the doubles we are operating on ...
user avatar
1vote
1answer
52views

How to get the finest possible precision in JavaScript?

I'm trying to determine the smallest possible non-negative, non-zero value available in a JavaScript number for addition and subtraction. If necessary then the smallest possible number for addition ...
user avatar
  • 2,326
1vote
1answer
66views

JavaScript: Decimal part of a float is caught by integer regex

I am trying to differentiate if a user's input is a float or an integer. However, when a user enters a float, the whole input is considered a float but the decimal part of it is considered an integer, ...
user avatar
  • 21
1vote
0answers
85views

When does JavaScript use `two's complement` format?

Two's complement is used to represent signed integers, however 64bit IEEE754 float can also represent signed integers. Since the latter is the format in which JavaScript stores its numbers, I'm ...
user avatar
1vote
0answers
90views

Why might IE give a wrong result for Math.floor()

I've got logs from my client-side app which show IE occasionally having trouble with Math.floor. Here's an example of one of these logs: var d = 1442167625975 d / 16 = 90135476623.4375 Math.floor(d ...
user avatar
  • 1,239
1vote
1answer
448views

Reading C++ float value from buffer in Javascript

currently i'm working on some client(html5)<->node.js<->server (c++) project. Server sends me a structured data (standard C++ structs). I have no problem with reading chars, arrays and int32's ...
user avatar
1vote
1answer
2kviews

convert floating point number to RGB or HEX

I have the following floating point number 8.8489784e-39 extracted from the PCD DATA FORMAT which represents the RGB value of one point of the point cloud. I have to convert this to either HEX ...
user avatar
  • 3,814
1vote
1answer
529views

How do I get the exact, floating point width of an element?

I am trying to get the exact width of a table cell. In IE9's developer toolbar, the Layout tab shows that it has a width of 203.68px, but element.clientWidth and other methods all report the rounded ...
user avatar
0votes
1answer
122views

Generating unbiased random float numbers [0, 1) in JavaScript

I found that the distribution of mantissa bits of random floats numbers generated by javascript Math.random() is biased. The least significant bit is always 0, and the right bits are more even than ...
user avatar
  • 2,018
0votes
0answers
26views

Does anyone know how best to cope with floating point errors in the FFT calculation process?

Does anyone know how best to cope with floating point errors in the FFT calculation process? The problem is that these errors accumulate because of all the summations. They are especially annoying ...
user avatar
0votes
0answers
42views

Dealing with floating point precision in power function

I am having trouble coming up with a class to deal with floating point precision, specifically the pow program which generates the nth power of a number (the base or exponent can be negative or exist ...
user avatar
0votes
1answer
39views

Floating points in big numbers from a select list (js)

I have 2 select lists in html, place to enter a number and second place to show the result. It works like a conversion, so when you choose milimeters from the first list and meters from the second it ...
user avatar
  • 71
0votes
0answers
40views

Binary Floating point to decimal JavaScript function

To answer the original question that was closed due to an invalid moderation. Binary Floating point to decimal JavaScript fucntion This function below basically does the math based on the floating ...
user avatar
0votes
2answers
54views

Parse string to floating point number

I'm struggling trying to parse some string that I'm reading from a txt file. When I parse it using parseFloat() it doesn't work as expected, for example when I log a "45.64" parsed to float ...
user avatar
0votes
0answers
31views

does javascript equality for 2.0 === 2 - hold true for all browsers

I want to write a function to trim additional decimal 0's from floating point numbers like 2.0, 5.00 but not 3.04. so I wrote this: const trimZeroFromDecimal = value => parseInt(value) === value ...
user avatar
  • 1,265
0votes
3answers
105views

How do I extract an individual number from a float

I need to extract individual numbers from a float without turning the float into a string but have no idea how to do that. I'm thinking of something like substr. but for a number.
user avatar
  • 1
0votes
0answers
50views

Convert float bytes to string

I have an application were I need to transmit simple data like floats as a string to a server and back. Converting those floats into the decimal format and vice versa works but probably isn't ...
user avatar
0votes
1answer
47views

Distributing currency/number across multiple items in array

So I have a sort of payment splitter and I'm struggling with floating point operations. I'm using currency.js to handle the maths and have jQuery installed too, though probably won't use it in the ...
user avatar
0votes
1answer
43views

My cash register function always forget to register the last cent whenever the price or the cash given is not an interger

This while loop essentially finds what's the best unit of the currency to give back to the customer considering what is in the drawer and adds it to the change array. while (difference > 0) { ...
user avatar
  • 15
0votes
0answers
45views

GeoJSON. JS/NodeJS, float number. Incorrect mid point of Line (Axy Bxy)

GEO JSON Point A: {124.8046875, -22.79643932091949} Point B: {143.26171875, -23.563987128451217} Need to find mid point of this line: MidPoint X: (124.8046875 + 143.26171875) / 2 = 134.033203125 ...
user avatar
0votes
0answers
36views

Floating point multiplication in JavaScript

I tried the below JavaScript in Chrome console: // Case 1 0.11 * 10000 // 1100 0.14 * 10000 // 1400.000...2 // Both of 0.11 and 0.14 are infinite in binary number // Case 2 0.14 * 10 // 1.4000...1 0....
user avatar
  • 462
0votes
0answers
31views

Vue getter/setter behavior showing unexpected side effect when * and / by 100

I have a getter and setter function in a Vue component that ideally would allow the user to type in percentages as whole numbers and my application use them as decimals. Here is the getter/setter: ...
user avatar
  • 724
0votes
0answers
34views

Adding javascript numbers does not have effect

let x = -75; let b = -7.105427357601002e-15; x = x + b; alert(x); Alert prints -75. Why? What is the solution to avoid this error?
user avatar
0votes
1answer
414views

Testing floating point logic using chai-almost and sinon `calledWithMatch`

I have a test case which is failing because a value being tested is off by Number.EPSILON. I understand why this is happening and believe I need to alter my test case so that it tolerates this ...
user avatar
0votes
0answers
126views

Will someone ever be faced with wrong results of simple arithmetic operations on floating point numbers if using rounding?

Note: numbers which I'm talking about are currency, thus don't have more than two digits in fractional part. I've tested 4 libraries (Decimal.js, Numeral.js, Big.js and Math.js) and a simple plain ...
user avatar
  • 8,108
0votes
1answer
45views

Adding up floating numbers in javascript

I was wondering about the way JavaScript handles floating points. I.e. suppose I set 5 variables to (no decimal numbers): var a = 1/2, b = 1/3, c = 1/12, d = 1/12, e = 0; Is the sum of these ...
user avatar
0votes
1answer
95views

Does a hardcode float number x.yz round to the same value as xyz/100?

According to Is floating point math broken?, 0.1 would be rounded up to some value because 1/10 cannot be represented exactly. But my question is, how would a hardcoded decimal round up? Does var a=...
user avatar
  • 3,918
0votes
0answers
758views

18 digits after the decimal point in JavaScript

I have a number 0.780000000000000016 (18 digits after decimal point, type 'number') I need to convert it to string. If I make 0.780000000000000016.toPrecision(18) it returns "0.780000000000000027" ...
user avatar
  • 275
0votes
1answer
61views

log base 2 precision error

I want to get mantissa of floating point number in JavaScript. So I get exponent part of floating point number first. By mathematics, exponent is floor(log2(x)). However, There is no log2 function ...
user avatar
0votes
1answer
86views

Issue with input number when number has .0

Got issue when trying to validate the input type number. I want to accept integers only. Don't want to accept float (it almost works). Got issue with number like 1.0, 3.0, but not with 1.1 When ...
user avatar

15 30 50 per page