Questions tagged [scientific-computing]
Use this tag for questions about using computers in science. Many of these questions also fit on scicomp.stackexchange.com.
702
questions
137votes
11answers
225kviews
What can MATLAB do that R cannot do? [closed]
I often hear people complain how expensive MATLAB licenses are. Then I wonder why they don't just use Octave or R. But is the latter right? Can you use R to replace MATLAB?
- 60.6k
127votes
6answers
215kviews
binning data in python with scipy/numpy
is there a more efficient way to take an average of an array in prespecified bins? for example, i have an array of numbers and an array corresponding to bin start and end positions in that array, and ...
user248237
93votes
6answers
111kviews
Interactive large plot with ~20 million sample points and gigabytes of data
I have got a problem (with my RAM) here: it's not able to hold the data I want to plot. I do have sufficient HD space. Is there any solution to avoid that "shadowing" of my data-set?
Concretely I ...
- 6,726
91votes
20answers
8kviews
Practices for programming in a scientific environment? [closed]
Background
Last year, I did an internship in a physics research group at a university. In this group, we mostly used LabVIEW to write programs for controlling our setups, doing data acquisition and ...
73votes
10answers
20kviews
F# performance in scientific computing
I am curious as to how F# performance compares to C++ performance? I asked a similar question with regards to Java, and the impression I got was that Java is not suitable for heavy numbercrunching.
...
- 48.4k
65votes
5answers
41kviews
True or false output based on a probability
Is there a standard function for Python which outputs True or False probabilistically based on the input of a random number from 0 to 1?
example of what I mean:
def decision(probability):
......
- 9,462
63votes
7answers
114kviews
Plotting with C# [closed]
C# seems to show some promise for scientific computing, but I found very little about one plotting 2D graphs, which is very important both for science student and scientists.
Is there a reliable, ...
Shinka
57votes
2answers
55kviews
Principal components analysis using pandas dataframe
How can I calculate Principal Components Analysis from data in a pandas dataframe?
- 589
51votes
6answers
33kviews
Is the order of a Python dictionary guaranteed over iterations?
I'm currently implementing a complex microbial food-web in Python using SciPy.integrate.ode. I need the ability to easily add species and reactions to the system, so I have to code up something quite ...
- 58.2k
50votes
4answers
34kviews
multithreaded blas in python/numpy
I am trying to implement a large number of matrix-matrix multiplications in Python. Initially, I assumed that NumPy would use automatically my threaded BLAS libraries since I built it against those ...
- 918
47votes
17answers
15kviews
The reading list for scientific programmer [closed]
I am working to become a scientific programmer. I have enough background in Math and Stat but rather lacking on programming background. I found it very hard to learn how to use a language for ...
- 4,711
36votes
10answers
37kviews
Open source alternative to MATLAB's fmincon function? [closed]
Is there an open-source alternative to MATLAB's fmincon function for constrained linear optimization? I'm rewriting a MATLAB program to use Python / NumPy / SciPy and this is the only function I haven'...
- 70.4k
35votes
4answers
4kviews
How to test scientific software?
I'm convinced that software testing indeed is very important, especially in science. However, over the last 6 years, I never have come across any scientific software project which was under regular ...
- 1,490
32votes
5answers
37kviews
Numercially stable softmax
Is there a numerically stable way to compute softmax function below?
I am getting values that becomes Nans in Neural network code.
np.exp(x)/np.sum(np.exp(y))
- 8,586
31votes
4answers
27kviews
sigmoidal regression with scipy, numpy, python, etc
I have two variables (x and y) that have a somewhat sigmoidal relationship with each other, and I need to find some sort of prediction equation that will enable me to predict the value of y, given any ...
- 311
27votes
5answers
3kviews
Is Haskell appropriate for long-running applications?
I think that Haskell is a beautiful language, and judging by the benchmarks, its implementations can generate fast code.
However, I am wondering if it is appropriate for long-running applications, ...
- 10.8k
26votes
7answers
5kviews
Scientific math with functional languages?
Are there any serious scientific math libraries made with functional programming languages? From the very nature of functional languages one would think that they are particularly suitable for math, ...
- 35.5k
25votes
3answers
13kviews
Javascript and Scientific Processing? [closed]
Matlab, R, and Python are powerful but either costly or slow for some data mining work I'd like to do. I'm considering using Javascript both for
speed, good visualization libraries, and to be able ...
- 776
24votes
10answers
3kviews
Where to find programming projects that help science? [closed]
I would like to work on a programming project in my spare time and would like to know
if there is a project where I can help the science community in some way?
- 1,808
24votes
4answers
17kviews
Spectrogram C++ library [closed]
For my current project in C++ / Qt I need a library (LGPL is preferred) which can calculate a spectrogram from a signal ( basically an array of doubles ). I already use Qwt for the GUI part.
Any ...
- 12.4k
23votes
1answer
29kviews
Logarithm Calculation with Windows 7 Calculator [closed]
I would like to use the Windows Calculator in Scientific Mode in order solve a very basic Logarithm equation but, unfortunately, I couldn't do that.
Here is the problem:
log_5 125=?
Thank you very ...
- 331
23votes
5answers
21kviews
How much of NumPy and SciPy is in C?
Are parts of NumPy and/or SciPy programmed in C/C++?
And how does the overhead of calling C from Python compare to the overhead of calling C from Java and/or C#?
I'm just wondering if Python is a ...
user93202
23votes
6answers
8kviews
Scientific libraries for Lua? [closed]
Are there any scientific packages for Lua comparable to Scipy?
- 32.1k
23votes
9answers
5kviews
Are functional programming languages good for practical tasks? [closed]
It seems to me from my experimenting with Haskell, Erlang and Scheme that functional programming languages are a fantastic way to answer scientific questions. For example, taking a small set of data ...
- 3,936
21votes
3answers
17kviews
Python distributions and environments for scientific computing
I apologize upfront if this question is too broad. I come from the MATLAB world and have relatively little experience with Python.
After having spent some time reading about several Python-based ...
- 81.7k
21votes
4answers
34kviews
Which IDE for scientific computing and plotting in Python? [closed]
I am currently using R for all my scientific computing and plotting, but I would like to explore Python. I have been using RStudio as an IDE for R, which as an IDE fulfills 100% of my need for ...
- 9,429
21votes
4answers
2kviews
what changes when your input is giga/terabyte sized?
I just took my first baby step today into real scientific computing today when I was shown a data set where the smallest file is 48000 fields by 1600 rows (haplotypes for several people, for ...
- 3,077
20votes
4answers
18kviews
Integrating a multidimensional integral in scipy
Motivation: I have a multidimensional integral, which for completeness I have reproduced below. It comes from the computation of the second virial coefficient when there is significant anisotropy:
...
- 76.7k
19votes
5answers
17kviews
Python module for multiple variable global optimization [closed]
I have been looking for a python module that implements the common techniques of global optimization (finding the global minimum of a function in N dimensions) without success.
If you heard about a ...
- 13.7k
18votes
8answers
1kviews
Reproducibility in scientific programming
Along with producing incorrect results, one of the worst fears in scientific programming is not being able to reproduce the results you've generated. What best practices help ensure your analysis is ...
- 73.6k
18votes
1answer
2kviews
How to organize a set of scientific experiments using Git
I'm running experiments on a model, with a workflow like this:
I work in a model (a software in Python)
I change some parameters and run an experiment
Then, I will store the results of the experiment ...
- 21.8k
16votes
4answers
13kviews
Writing a faster Python physics simulator
I have been playing around with writing my own physics engine in Python as an exercise in physics and programming. I started out by following the tutorial located here. That went well, but then I ...
- 745
16votes
5answers
5kviews
(Re)Starting with C++ (for scientific computing)
I have a fair hang of programming in various languages. I have been implementing my codes for research using MATLAB (during the past few months) and for the first time really noticed the difference in ...
user59634
16votes
4answers
13kviews
Cell segmentation and fluorescence counting in Python [closed]
How can I segment cells from an image taken on a microscope, along the lines of what was done here in Matlab?
http://blogs.mathworks.com/steve/2006/06/02/cell-segmentation/
Also, if I take multiple ...
user248237
15votes
4answers
15kviews
What is a good free (open source) BLAS/LAPACK library for .net (C#)? [closed]
I have a project written in C# where I need to do various linear algebraic operations on matrices (like LU-factorization).
Since the program is mainly a prototype created to confirm a theory, a C# ...
- 14.1k
15votes
3answers
2kviews
Sparse Matrix Libraries for Ruby
I'm looking for a Sparse Matrix library I can use from Ruby. I'm currently using the GNU Scientific Library bindings provided by the "gsl" gem, but my application would be better optimized if I used a ...
- 560
14votes
3answers
12kviews
Interpolation in SciPy: Finding X that produces Y
Is there a better way to find which X gives me the Y I am looking for in SciPy? I just began using SciPy and I am not too familiar with each function.
import numpy as np
import matplotlib.pyplot as ...
- 1,228
14votes
4answers
6kviews
What are your "must-have" Python Packages for Finance?
With the recent SEC proposal requiring that most Asset-Backed Securities issuers file a python computer program to document the flow of funds (or waterfall) provisions of the transaction, I thought it ...
14votes
2answers
3kviews
Scientific Programming Stack for Clojure
I currently use Python for most of my programming, but I'm interested in learning Clojure. What libraries do I need to reproduce the functionality I have with scipy, numpy, and matplotlib? Is there ...
- 2,659
14votes
1answer
3kviews
Floating-point optimizations - guideline
The majority of scientific computing problems that we need solve by implementing a particular algorithm in C/C++ demands accuracy that are much lower than double precision. For example, 1e-6, 1e-7 ...
- 2,326
13votes
2answers
2kviews
Qt vs Visual Studio for scientific computing [closed]
I am embedded programmer, and have no experience with desktop programming. In my project have to display and calculate no of physical parameter like pH, temp.
Please suggest me which way to go: Qt or ...
- 119
13votes
1answer
12kviews
Plotting log-binned network degree distributions
I have often encountered and made long-tailed degree distributions/histograms from complex networks like the figures below. They make the heavy end of these tails, well, very heavy and crowded from ...
- 2,082
13votes
1answer
3kviews
multinomial pmf in python scipy/numpy
Is there a built-in function in scipy/numpy for getting the PMF of a Multinomial? I'm not sure if binom generalizes in the correct way, e.g.
# Attempt to define multinomial with n = 10, p = [0.1, 0.1,...
user248237
12votes
12answers
8kviews
Modern language with the advantages of FORTRAN?
I've been working with a lot of FORTRAN 77 lately, and though I had reservations at first, I now see some great qualities of the language. It is:
Simple to learn (no object-oriented or functional-...
- 7,483
12votes
2answers
54kviews
How to represent scientific notation in C
How do I represent extremely large or small numbers in C with a certain amount of significant figures. For example, if I want to do calculations on 1.54334E-34, how could I do this. Also, is this ...
- 12.5k
12votes
2answers
3kviews
Tracing Python warnings/errors to a line number in numpy and scipy
I am getting the error:
Warning: invalid value encountered in log
From Python and I believe the error is thrown by numpy (using version 1.5.0). However, since I am calling the "log" function in ...
user248237
12votes
4answers
2kviews
Methods for entering equations while programming in C/C++ , Python or Fortran
I am writing a code which had long mathematical equations with many trigonometric and other identities. Is there a way of visualising the same expression in latex and making a C or python expression ...
- 177
12votes
5answers
8kviews
How to deal with underflow in scientific computing?
I am working on probabilistic models, and when doing inference on those models, the estimated probabilities can become very small. In order to avoid underflow, I am currently working in the log domain ...
- 1,463
12votes
1answer
11kviews
statsmodels ARIMA.fit: Hide output
It seems whenever I run ARIMA.fit(), I always get a stdout from the kalman filter:
## -- End pasted text --
RUNNING THE L-BFGS-B CODE
* * *
Machine precision = 2.220D-16
N = ...
- 18.1k
12votes
1answer
3kviews
Why are there no BLAS routines for addition and subtraction
In BLAS there are routines like
dscal scale a vector by a constant
dinit initialize a vector with given value
daxpy perform y = a*x + y
and so on. But there are apparently no routines ...
- 4,851