Questions tagged [floating-point]

Floating point numbers are approximations of real numbers that can represent larger ranges than integers but use the same amount of memory, at the cost of lower precision. If your question is about small arithmetic errors (e.g. why does 0.2 + 0.1 equal 0.300000001?) or decimal conversion errors, please read the "info" page linked below before posting.

1,634 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
-2votes
2answers
100views

Python 3: Is this a float or a list?

I have a class which has the attribute 'latest_level' and this should be a floating point number. I then have a method which includes latest_level in an equation to return another floating point ...
user avatar
-2votes
2answers
164views

How to check the input format in java

i need to check whether the number input is in decimal format or in floating point format in java coding. in simple terms how would this check be possible?
user avatar
  • 1
-2votes
2answers
1kviews

avoiding zero float division in loops - python

sorry i am a bit of a newbie with programming but I am getting a float division error in a simple loop which I am not sure how to rectify. Here is a code in python 2.7 import random N = 100 A = [] ...
user avatar
-2votes
1answer
210views

IEEE-754 single precision 1-bit sign 8-bit exponent 23-bit fraction, What is the binary representation of 0.25*2^(-128)?

IEEE-754 single precision 1-bit sign 8-bit exponent 23-bit fraction, What is the binary representation of 0.25*2^(-128)=2^(-130)? by using the formula: exponent-bias=log(given number) and also ...
user avatar
  • 17
-2votes
1answer
88views

Division overflow using pow()

Q = (a_i + b_i) / (2^s) -10^10 ≤ s ≤ 10^10 1 ≤ a_i, b_i ≤ 10^9 It is guaranteed that -10^10 ≤ Q ≤  10^10. Here s,a_i,b_i are integers and Q is a decimal no. When we calculate Q, there is overflow ...
user avatar
  • 13
-2votes
2answers
432views

PHP float number with decimal point rounding

I am calculating few numbers (sales_total, service_charge) to get 7% GST. What I get is number with decimal point. Example 1: $sales_total = 207.50; $service_charge = 20.75; $gst ...
user avatar
  • 3,470
-2votes
2answers
848views

MySQL rounding of Float Data type

set up a table in mySQL with a column FLOAT(2,2), then tried to insert 10 and it diaplayed 0.99?
user avatar
-2votes
3answers
4kviews

TypeError: 'float' object is not subscriptable while saving a dict

I keep getting this error while trying to use pickle to save a dictionary, can anyone help? Traceback (most recent call last): File "/home/euan/Documents/Python/Task 1.04.py", line 31, in <...
user avatar
-2votes
1answer
1kviews

Performing boundary value testing on float range

I want to perform boundary value analysis on a program which takes 3 float variables as input and returns the largest out of them. There's no range specified hence I impose the range to be the one ...
user avatar
  • 57
-2votes
2answers
360views

Floating point number and range of floating point numbers that can be represented by a string

refer to the string of bits 010011110110 We also assume that when the number is stored as a floating point (real) number, 6 of the 12 bits are reserved for the mantissa (or significand) If the ...
user avatar
-2votes
1answer
253views

Using unsigned long long instead of float to prevent wrapping

I have an application in which I am using a certain class that has a method to return current elapsed time to me. unsigned long long getCurrentElapsedTime(){ } Unsigned long long is used instead of ...
user avatar
  • 5,594
-2votes
1answer
32views

Can an integer divided by another integer really become a float? Or is this a bug?

I had this: $test = (1590962400 / 1000); var_dump($test); It outputs: float(1590962.4) This really puzzled me, because my math knowledge seemed to tell me that two "solid" numbers like ...
user avatar
  • 1
-2votes
2answers
236views

Is there a way to convert 0.15 to 15

I want to convert a floating point number like 0.15 -> 15 or 0.4 -> 4 My idea is to Multiply a number with amount of floating point places. Floor it. Cast it to (int). Can't figure how to do ...
user avatar
-2votes
1answer
135views

Remove decimal point from any arbitrary decimal number

I am writing Python3 code which requires me to simpy remove the decimal. I tried the most simple thing I could think of num = int(str(float(x)).replace('.', '')) where x is generated using x = a*x*(1-...
user avatar
  • 785
-2votes
3answers
2kviews

How do I remove decimals in Python 3?

Here is the code I'm using to prompt user to input wall's width in feet: paintColors = { "red": 35, "blue": 25, "green": 23 } wallHeight = float(input('Enter wall height (feet)): \n')) ...
user avatar
-2votes
1answer
121views

C# NumericUpDown with IEEE754 Single Precision Resolution

I want to be able to construct a numericUpDown field in C# where on each arrow click the value in the field increases by one resolution. For example; Let's say my value is 20.0, and it's IEEE754 ...
user avatar
-3votes
4answers
74views

How to round all elements of a list using a list comprehension?

Considering the following list of float values aList = ['3.32', '1.08', '4.23'] How do I use a list comprehension to round it and obtain the following [3.0, 1.0, 4.0]?
user avatar
-3votes
2answers
110views

How to square any non-integer number without pow() and multiplication?

I want to replace pow with something other than multiplication. How can i do it? THIS PROGRAM SEARCHES FOR THE ROOT OF THE SUM OF SQUARES OF ENTERED NUMBERS: #include <stdio.h> #include <math....
user avatar
-3votes
1answer
83views

Print out float/double variables for hardware verification

I have the following scenario. A algorithm with A*B+C*D in floating-point variables calculation is implemented in C++. In order to implement it in hardware, we need to print out the A, B, C, D and ...
user avatar
-3votes
2answers
386views

Java program to find integer root of a quadratic equation

So, here is my requirement. If a quadratic equation has two roots(an int & a float), I want to take only integer value for further manipulation. I can't figure it out how it's made. Can anyone ...
user avatar
-3votes
1answer
888views

How to find 32 bit floating point representation of base 10 number?

I had this question on a quiz today. Can anyone explain to me the solution and how to even find the solution? What is the 32-bit floating point representation of the base-10 number 0.53125? In the ...
user avatar
-3votes
1answer
111views

C Help! Floating Points

I have this question: We claim that it is unsafe to use equality testing with floating point numbers because the arithmetic operations introduce rounding errors which mean that two numbers which ...
user avatar
-3votes
1answer
3kviews

How to write float values to a text file in Java

I am reading files from a GRD file: File stockInputFile = new File("C://MAX.GRD"); I want to write them to a text file: File StockOutputFile = new File("C://StockOut.txt"); Am getting some Korean ...
user avatar
-3votes
4answers
66views

The converted floating to decimal point?

In case x = 0.898558 #float with zero prefix I tried from decimal import Decimal x = 0.898558 print(Decimal(x)) Output: 0.898557999999999967855046634213067591190338134765625 I think i can do ...
user avatar
-3votes
1answer
36views

I'm getting this error but I tried everything. TypeError: can't multiply sequence by non-int of type 'float'

I'm having an issue with running this simple calculator. Every time I run this program, I get this error: f = a * l TypeError: can't multiply sequence by non-int of type 'float' Does anyone ...
user avatar
-4votes
1answer
796views

Expected type int got float in Pycharm

sub1=int(input("Enter the marks of the first subject: ")) sub2=int(input("Enter the marks of the second subject: ")) sub3=int(input("Enter the marks of the third subject: ")) sub4=int(...
user avatar
-4votes
1answer
430views

Min and Max Value c++

So I have some code that i am writing for an assignment and I am stuck on finding the min and max values from 3 floats. I am currently getting the error /usr/include/c++/4.9/bits/predefined_ops.h:...
user avatar
  • 1
-4votes
2answers
186views

to write a function that tests whether a Point is in a Rectangle

the question is as follows: write and test a program with the following features. Firstly, defines a new structured type called Point, is represented with floats for the x and y values . Also, ...
user avatar
-4votes
1answer
91views

Floating Point Exception from my for loop that is not initialized correctly

I keep getting a floating point expection on my program that is trying to determine if a number is prime or not in C. I used this loop: for(i=1:i<num/2:i++) (if num%i==0) then it is prime. ...
user avatar
  • 1
-4votes
3answers
204views

How to turn a float into an int

have a decimal float from a user input calculation, say 2.856 that I need to turn into the integer 2. Note, this has to be done in the code itself and cannot simply be output with %.0f; I need it for ...
user avatar
-4votes
1answer
71views

What is the number N represented in floating point by the binary string of 32 bits whose hexadecimal representation is 8A000000?

What is the number N represented in floating point by the binary string of 32 bits whose hexadecimal representation is 8A000000? Please help me. How can I find it?
user avatar
-5votes
1answer
420views

why Float 999999.99 auto round to 1000000.0, but 99999.99 ,99999999, wont auto round

java. i have known "A float is generally subject to round-offs in its 8th significant digit",and using double will avoid that problem. can anyone tell me why it will round-offs in its 8th ...
user avatar
-5votes
1answer
982views

Confused how to read assembly instruction

I have some questions dealing with reading assembly instruction. I know the basic instruction syntax, but it gets confusing when some of the register comes with some address in front of it, or the ...
user avatar
  • 219
-6votes
3answers
4kviews

"The Trip" Programming Challenges

Here's the problem. A group of students are members of a club that travels annually to different locations. Their destinations in the past have included Indianapolis, Phoenix, Nashville, Philadelphia,...
user avatar
  • 11

15 30 50 per page
1
29 30 31 32
33