# Questions tagged [scientific-computing]

Use this tag for questions about using computers in science. Many of these questions also fit on scicomp.stackexchange.com.

702
questions

137votes

11answers

225kviews

### What can MATLAB do that R cannot do? [closed]

I often hear people complain how expensive MATLAB licenses are. Then I wonder why they don't just use Octave or R. But is the latter right? Can you use R to replace MATLAB?

127votes

6answers

215kviews

### binning data in python with scipy/numpy

is there a more efficient way to take an average of an array in prespecified bins? for example, i have an array of numbers and an array corresponding to bin start and end positions in that array, and ...

93votes

6answers

111kviews

### Interactive large plot with ~20 million sample points and gigabytes of data

I have got a problem (with my RAM) here: it's not able to hold the data I want to plot. I do have sufficient HD space. Is there any solution to avoid that "shadowing" of my data-set?
Concretely I ...

91votes

20answers

8kviews

### Practices for programming in a scientific environment? [closed]

Background
Last year, I did an internship in a physics research group at a university. In this group, we mostly used LabVIEW to write programs for controlling our setups, doing data acquisition and ...

73votes

10answers

20kviews

### F# performance in scientific computing

I am curious as to how F# performance compares to C++ performance? I asked a similar question with regards to Java, and the impression I got was that Java is not suitable for heavy numbercrunching.
...

65votes

5answers

41kviews

### True or false output based on a probability

Is there a standard function for Python which outputs True or False probabilistically based on the input of a random number from 0 to 1?
example of what I mean:
def decision(probability):
......

63votes

7answers

114kviews

### Plotting with C# [closed]

C# seems to show some promise for scientific computing, but I found very little about one plotting 2D graphs, which is very important both for science student and scientists.
Is there a reliable, ...

57votes

2answers

55kviews

### Principal components analysis using pandas dataframe

How can I calculate Principal Components Analysis from data in a pandas dataframe?

51votes

6answers

33kviews

### Is the order of a Python dictionary guaranteed over iterations?

I'm currently implementing a complex microbial food-web in Python using SciPy.integrate.ode. I need the ability to easily add species and reactions to the system, so I have to code up something quite ...

50votes

4answers

34kviews

### multithreaded blas in python/numpy

I am trying to implement a large number of matrix-matrix multiplications in Python. Initially, I assumed that NumPy would use automatically my threaded BLAS libraries since I built it against those ...

47votes

17answers

15kviews

### The reading list for scientific programmer [closed]

I am working to become a scientific programmer. I have enough background in Math and Stat but rather lacking on programming background. I found it very hard to learn how to use a language for ...

36votes

10answers

37kviews

### Open source alternative to MATLAB's fmincon function? [closed]

Is there an open-source alternative to MATLAB's fmincon function for constrained linear optimization? I'm rewriting a MATLAB program to use Python / NumPy / SciPy and this is the only function I haven'...

35votes

4answers

4kviews

### How to test scientific software?

I'm convinced that software testing indeed is very important, especially in science. However, over the last 6 years, I never have come across any scientific software project which was under regular ...

32votes

5answers

37kviews

### Numercially stable softmax

Is there a numerically stable way to compute softmax function below?
I am getting values that becomes Nans in Neural network code.
np.exp(x)/np.sum(np.exp(y))

31votes

4answers

27kviews

### sigmoidal regression with scipy, numpy, python, etc

I have two variables (x and y) that have a somewhat sigmoidal relationship with each other, and I need to find some sort of prediction equation that will enable me to predict the value of y, given any ...

27votes

5answers

3kviews

### Is Haskell appropriate for long-running applications?

I think that Haskell is a beautiful language, and judging by the benchmarks, its implementations can generate fast code.
However, I am wondering if it is appropriate for long-running applications, ...

26votes

7answers

5kviews

### Scientific math with functional languages?

Are there any serious scientific math libraries made with functional programming languages? From the very nature of functional languages one would think that they are particularly suitable for math, ...

25votes

3answers

13kviews

### Javascript and Scientific Processing? [closed]

Matlab, R, and Python are powerful but either costly or slow for some data mining work I'd like to do. I'm considering using Javascript both for
speed, good visualization libraries, and to be able ...

24votes

10answers

3kviews

### Where to find programming projects that help science? [closed]

I would like to work on a programming project in my spare time and would like to know
if there is a project where I can help the science community in some way?

24votes

4answers

17kviews

### Spectrogram C++ library [closed]

For my current project in C++ / Qt I need a library (LGPL is preferred) which can calculate a spectrogram from a signal ( basically an array of doubles ). I already use Qwt for the GUI part.
Any ...

23votes

1answer

29kviews

### Logarithm Calculation with Windows 7 Calculator [closed]

I would like to use the Windows Calculator in Scientific Mode in order solve a very basic Logarithm equation but, unfortunately, I couldn't do that.
Here is the problem:
log_5 125=?
Thank you very ...

23votes

5answers

21kviews

### How much of NumPy and SciPy is in C?

Are parts of NumPy and/or SciPy programmed in C/C++?
And how does the overhead of calling C from Python compare to the overhead of calling C from Java and/or C#?
I'm just wondering if Python is a ...

23votes

6answers

8kviews

### Scientific libraries for Lua? [closed]

Are there any scientific packages for Lua comparable to Scipy?

23votes

9answers

5kviews

### Are functional programming languages good for practical tasks? [closed]

It seems to me from my experimenting with Haskell, Erlang and Scheme that functional programming languages are a fantastic way to answer scientific questions. For example, taking a small set of data ...

21votes

3answers

17kviews

### Python distributions and environments for scientific computing

I apologize upfront if this question is too broad. I come from the MATLAB world and have relatively little experience with Python.
After having spent some time reading about several Python-based ...

21votes

4answers

34kviews

### Which IDE for scientific computing and plotting in Python? [closed]

I am currently using R for all my scientific computing and plotting, but I would like to explore Python. I have been using RStudio as an IDE for R, which as an IDE fulfills 100% of my need for ...

21votes

4answers

2kviews

### what changes when your input is giga/terabyte sized?

I just took my first baby step today into real scientific computing today when I was shown a data set where the smallest file is 48000 fields by 1600 rows (haplotypes for several people, for ...

20votes

4answers

18kviews

### Integrating a multidimensional integral in scipy

Motivation: I have a multidimensional integral, which for completeness I have reproduced below. It comes from the computation of the second virial coefficient when there is significant anisotropy:
...

19votes

5answers

17kviews

### Python module for multiple variable global optimization [closed]

I have been looking for a python module that implements the common techniques of global optimization (finding the global minimum of a function in N dimensions) without success.
If you heard about a ...

18votes

8answers

1kviews

### Reproducibility in scientific programming

Along with producing incorrect results, one of the worst fears in scientific programming is not being able to reproduce the results you've generated. What best practices help ensure your analysis is ...

18votes

1answer

2kviews

### How to organize a set of scientific experiments using Git

I'm running experiments on a model, with a workflow like this:
I work in a model (a software in Python)
I change some parameters and run an experiment
Then, I will store the results of the experiment ...

16votes

4answers

13kviews

### Writing a faster Python physics simulator

I have been playing around with writing my own physics engine in Python as an exercise in physics and programming. I started out by following the tutorial located here. That went well, but then I ...

16votes

5answers

5kviews

### (Re)Starting with C++ (for scientific computing)

I have a fair hang of programming in various languages. I have been implementing my codes for research using MATLAB (during the past few months) and for the first time really noticed the difference in ...

16votes

4answers

13kviews

### Cell segmentation and fluorescence counting in Python [closed]

How can I segment cells from an image taken on a microscope, along the lines of what was done here in Matlab?
http://blogs.mathworks.com/steve/2006/06/02/cell-segmentation/
Also, if I take multiple ...

15votes

4answers

15kviews

### What is a good free (open source) BLAS/LAPACK library for .net (C#)? [closed]

I have a project written in C# where I need to do various linear algebraic operations on matrices (like LU-factorization).
Since the program is mainly a prototype created to confirm a theory, a C# ...

15votes

3answers

2kviews

### Sparse Matrix Libraries for Ruby

I'm looking for a Sparse Matrix library I can use from Ruby. I'm currently using the GNU Scientific Library bindings provided by the "gsl" gem, but my application would be better optimized if I used a ...

14votes

3answers

12kviews

### Interpolation in SciPy: Finding X that produces Y

Is there a better way to find which X gives me the Y I am looking for in SciPy? I just began using SciPy and I am not too familiar with each function.
import numpy as np
import matplotlib.pyplot as ...

14votes

4answers

6kviews

### What are your "must-have" Python Packages for Finance?

With the recent SEC proposal requiring that most Asset-Backed Securities issuers file a python computer program to document the flow of funds (or waterfall) provisions of the transaction, I thought it ...

14votes

2answers

3kviews

### Scientific Programming Stack for Clojure

I currently use Python for most of my programming, but I'm interested in learning Clojure. What libraries do I need to reproduce the functionality I have with scipy, numpy, and matplotlib? Is there ...

14votes

1answer

3kviews

### Floating-point optimizations - guideline

The majority of scientific computing problems that we need solve by implementing a particular algorithm in C/C++ demands accuracy that are much lower than double precision. For example, 1e-6, 1e-7 ...

13votes

2answers

2kviews

### Qt vs Visual Studio for scientific computing [closed]

I am embedded programmer, and have no experience with desktop programming. In my project have to display and calculate no of physical parameter like pH, temp.
Please suggest me which way to go: Qt or ...

13votes

1answer

12kviews

### Plotting log-binned network degree distributions

I have often encountered and made long-tailed degree distributions/histograms from complex networks like the figures below. They make the heavy end of these tails, well, very heavy and crowded from ...

13votes

1answer

3kviews

### multinomial pmf in python scipy/numpy

Is there a built-in function in scipy/numpy for getting the PMF of a Multinomial? I'm not sure if binom generalizes in the correct way, e.g.
# Attempt to define multinomial with n = 10, p = [0.1, 0.1,...

12votes

12answers

8kviews

### Modern language with the advantages of FORTRAN?

I've been working with a lot of FORTRAN 77 lately, and though I had reservations at first, I now see some great qualities of the language. It is:
Simple to learn (no object-oriented or functional-...

12votes

2answers

54kviews

### How to represent scientific notation in C

How do I represent extremely large or small numbers in C with a certain amount of significant figures. For example, if I want to do calculations on 1.54334E-34, how could I do this. Also, is this ...

12votes

2answers

3kviews

### Tracing Python warnings/errors to a line number in numpy and scipy

I am getting the error:
Warning: invalid value encountered in log
From Python and I believe the error is thrown by numpy (using version 1.5.0). However, since I am calling the "log" function in ...

12votes

4answers

2kviews

### Methods for entering equations while programming in C/C++ , Python or Fortran

I am writing a code which had long mathematical equations with many trigonometric and other identities. Is there a way of visualising the same expression in latex and making a C or python expression ...

12votes

5answers

8kviews

### How to deal with underflow in scientific computing?

I am working on probabilistic models, and when doing inference on those models, the estimated probabilities can become very small. In order to avoid underflow, I am currently working in the log domain ...

12votes

1answer

11kviews

### statsmodels ARIMA.fit: Hide output

It seems whenever I run ARIMA.fit(), I always get a stdout from the kalman filter:
## -- End pasted text --
RUNNING THE L-BFGS-B CODE
* * *
Machine precision = 2.220D-16
N = ...

12votes

1answer

3kviews

### Why are there no BLAS routines for addition and subtraction

In BLAS there are routines like
dscal scale a vector by a constant
dinit initialize a vector with given value
daxpy perform y = a*x + y
and so on. But there are apparently no routines ...