Questions tagged [linear-algebra]
Linear Algebra is a core body of mathematics studying vector spaces and linear mappings between these spaces. Key topics include: matrices, vectors, simultaneous systems of linear equations, diagonal form, eigenvalues, eigenvectors, projections, points, lines, planes, hyperplanes, rotations and reflections.
4,038
questions
267votes
11answers
185kviews
What are the most widely used C++ vector/matrix math/linear algebra libraries, and their cost and benefit tradeoffs? [closed]
It seems that many projects slowly come upon a need to do matrix math, and fall into the trap of first building some vector classes and slowly adding in functionality until they get caught building a ...
- 16.5k
209votes
12answers
172kviews
"Cloning" row or column vectors
Sometimes it is useful to "clone" a row or column vector to a matrix. By cloning I mean converting a row vector such as
[1, 2, 3]
Into a matrix
[[1, 2, 3],
[1, 2, 3],
[1, 2, 3]]
or a ...
- 26.9k
112votes
10answers
67kviews
Math - mapping numbers
How do I map numbers, linearly, between a and b to go between c and d.
That is, I want numbers between 2 and 6 to map to numbers between 10 and 20... but I need the generalized case.
My brain is ...
- 5,797
91votes
2answers
44kviews
cocktail party algorithm SVD implementation ... in one line of code?
In a slide within the introductory lecture on machine learning by Stanford's Andrew Ng at Coursera, he gives the following one line Octave solution to the cocktail party problem given the audio ...
- 2,550
69votes
2answers
222kviews
Rotating a Vector in 3D Space
I am making an android project in opengl es that uses accelerometer to calculate change in specific axes and my aim is to rotate my spacecraft-like object's movement vector. The problem is that i can'...
- 2,257
62votes
7answers
125kviews
Python Inverse of a Matrix
How do I get the inverse of a matrix in python? I've implemented it myself, but it's pure python, and I suspect there are faster modules out there to do it.
- 215k
52votes
7answers
42kviews
C# linear algebra library
Is there stable linear algebra (more specifically, vectors, matrices, multidimensional arrays and basic operations on them) library for C#?
Search yielded a few open source libraries which are ...
- 7,865
51votes
3answers
34kviews
Understanding tensordot
After I learned how to use einsum, I am now trying to understand how np.tensordot works.
However, I am a little bit lost especially regarding the various possibilities for the parameter axes.
To ...
- 1,968
48votes
11answers
42kviews
Solving a linear equation
I need to programmatically solve a system of linear equations in C, Objective C, or (if needed) C++.
Here's an example of the equations:
-44.3940 = a * 50.0 + b * 37.0 + tx
-45.3049 = a * 43.0 + b * ...
- 49k
46votes
5answers
14kviews
Linear index upper triangular matrix
If I have the upper triangular portion of a matrix, offset above the diagonal, stored as a linear array, how can the (i,j) indices of a matrix element be extracted from the linear index of the array?
...
- 4,945
43votes
2answers
57kviews
whats the fastest way to find eigenvalues/vectors in python?
Currently im using numpy which does the job. But, as i'm dealing with matrices with several thousands of rows/columns and later this figure will go up to tens of thousands, i was wondering if there ...
- 7,771
40votes
3answers
84kviews
Algorithm for intersection of 2 lines?
I have 2 lines. Both lines containing their 2 points of X and Y. This means they both have length.
I see 2 formulas, one using determinants and one using normal algebra. Which would be the most ...
- 55.3k
38votes
4answers
25kviews
Sample from multivariate normal/Gaussian distribution in C++
I've been hunting for a convenient way to sample from a multivariate normal distribution. Does anyone know of a readily available code snippet to do that? For matrices/vectors, I'd prefer to use ...
- 6,157
37votes
8answers
29kviews
How expensive is it to compute the eigenvalues of a matrix?
How expensive is it to compute the eigenvalues of a matrix?
What is the complexity of the best algorithms?
How long might it take in practice if I have a 1000 x 1000 matrix? I assume it helps if ...
Frank
36votes
6answers
46kviews
Python (NumPy, SciPy), finding the null space of a matrix
I'm trying to find the null space (solution space of Ax=0) of a given matrix. I've found two examples, but I can't seem to get either to work. Moreover, I can't understand what they're doing to get ...
- 4,725
36votes
1answer
29kviews
Why does numpy.linalg.solve() offer more precise matrix inversions than numpy.linalg.inv()?
I do not quite understand why numpy.linalg.solve() gives the more precise answer, whereas numpy.linalg.inv() breaks down somewhat, giving (what I believe are) estimates.
For a concrete example, I am ...
- 14.4k
36votes
4answers
31kviews
Efficient & pythonic check for singular matrix
Working on some matrix algebra here. Sometimes I need to invert a matrix that may be singular or ill-conditioned. I understand it is pythonic to simply do this:
try:
i = linalg.inv(x)
except ...
- 2,319
35votes
3answers
5kviews
Efficient dot products of large memory-mapped arrays
I'm working with some rather large, dense numpy float arrays that currently reside on disk in PyTables CArrays. I need to be able to perform efficient dot products using these arrays, for example C = ...
- 66.8k
34votes
5answers
11kviews
What is SVD(singular value decomposition)
How does it actually reduce noise..can you suggest some nice tutorials?
shiva
34votes
3answers
33kviews
Numpy Adding two vectors with different sizes
If I have two numpy arrays of different sizes, how can I superimpose them.
a = numpy([0, 10, 20, 30])
b = numpy([20, 30, 40, 50, 60, 70])
What is the cleanest way to add these two vectors to produce ...
- 3,071
34votes
1answer
45kviews
Efficient element-wise multiplication of a matrix and a vector in TensorFlow
What would be the most efficient way to multiply (element-wise) a 2D tensor (matrix):
x11 x12 .. x1N
...
xM1 xM2 .. xMN
by a vertical vector:
w1
...
wN
to obtain a new matrix:
x11*w1 x12*w2 ... ...
- 30.6k
33votes
5answers
46kviews
Efficient 4x4 matrix inverse (affine transform)
I was hoping someone can point out an efficient formula for 4x4 affine matrix transform. Currently my code uses cofactor expansion and it allocates a temporary array for each cofactor. It's easy to ...
- 3,411
33votes
7answers
41kviews
Matrix power in R
Trying to compute the power of a matrix in R, I found that package expm implements the operator %^%.
So x %^% k computes the k-th power of a matrix.
> A<-matrix(c(1,3,0,2,8,4,1,1,1),nrow=3)
...
- 28.6k
31votes
5answers
6kviews
Is Google's Android OpenGL tutorial teaching incorrect linear algebra?
After helping another user with a question regarding the Responding to Touch Events Android tutorial, I downloaded the source code, and was quite baffled by what I saw. The tutorial seems to not be ...
- 34.7k
30votes
7answers
27kviews
How to create random orthonormal matrix in python numpy
Is there a method that I can call to create a random orthonormal matrix in python? Possibly using numpy? Or is there a way to create a orthonormal matrix using multiple numpy methods? Thanks.
- 411
30votes
6answers
44kviews
How to find linearly independent rows from a matrix
How to identify the linearly independent rows from a matrix? For instance,
The 4th rows is independent.
- 15.5k
30votes
6answers
123kviews
Transforming a row vector into a column vector in Numpy
Let's say I have a row vector of the shape (1, 256). I want to transform it into a column vector of the shape (256, 1) instead. How would you do it in Numpy?
- 2,413
30votes
1answer
1kviews
Haskell Linear Algebra Matrix Library for Arbitrary Element Types [closed]
I'm looking for a Haskell linear algebra library that has the following features:
Matrix multiplication
Matrix addition
Matrix transposition
Rank calculation
Matrix inversion is a plus
and has the ...
- 49.8k
28votes
1answer
32kviews
Most efficient way to solve a system of linear equations
I have an (n x n) symmetric matrix A and a (n x 1) vector B. Basically, I just need to solve Ax = b for x. The issue is that A is going to potentially be massive. So I'm looking for the most ...
- 535
28votes
5answers
28kviews
Equivalent of `polyfit` for a 2D polynomial in Python
I'd like to find a least-squares solution for the a coefficients in
z = (a0 + a1*x + a2*y + a3*x**2 + a4*x**2*y + a5*x**2*y**2 + a6*y**2 +
a7*x*y**2 + a8*x*y)
given arrays x, y, and z of ...
- 355
28votes
1answer
6kviews
Haskell linear algebra?
I am starting to test Haskell for linear algebra. Does anyone have any recommendations for the best package for this purpose? Any other good resources for doing basic matrix manipulation with ...
- 95.2k
27votes
2answers
17kviews
Can't understand the cost function for Linear Regression
I really can't understand the following equation, especially 1/(2m).
What's the purpose of this equation? And where does 1/(2m) came from?
J(theta_0, theta_1) = 1/(2m) * sum_(i=1)^m [ h_theta(x^i) -...
- 287
27votes
2answers
14kviews
Why does my translation matrix needs to be transposed?
I'm working on a small graphics engine using OpenGL and I'm having some issues with my translation matrix. I'm using OpenGL 3.3, GLSL and C++.
The situation is this: I have defined a small cube which ...
- 601
27votes
1answer
2kviews
Travelling salesman with a directional constraint
I am trying to order an array of 3D coordinates by their order along a path. A sample:
points = np.array([[ 0.81127451, 0.22794118, 0.52009804],
[ 0.62986425, 0.4546003 , 0....
- 5,541
26votes
2answers
17kviews
How to try-except an illegal matrix operation due to singularity in NumPy
In NumPy, I'm trying to use linalg to compute matrix inverses at each step of a Newton-Raphson scheme (the problem size is small intentionally so that we can invert analytically computed Hessian ...
- 68.8k
26votes
9answers
53kviews
Row-major vs Column-major confusion
I've been reading a lot about this, the more I read the more confused I get.
My understanding: In row-major rows are stored contiguously in memory, in column-major columns are stored contiguously in ...
- 4,973
25votes
2answers
32kviews
Fitting a line in 3D
Are there any algorithms that will return the equation of a straight line from a set of 3D data points? I can find plenty of sources which will give the equation of a line from 2D data sets, but none ...
- 819
25votes
4answers
10kviews
What does data.norm() < 1000 do in PyTorch?
I am following the PyTorch tutorial here.
It says that
x = torch.randn(3, requires_grad=True)
y = x * 2
while y.data.norm() < 1000:
y = y * 2
print(y)
Out:
tensor([-590.4467, 97.6760, ...
- 1,226
25votes
4answers
6kviews
How to compute scipy sparse matrix determinant without turning it to dense?
I am trying to figure out the fastest method to find the determinant of sparse symmetric and real matrices in python. using scipy sparse module but really surprised that there is no determinant ...
- 253
24votes
2answers
16kviews
scipy.linalg.eig return complex eigenvalues for covariance matrix?
The eigenvalues of a covariance matrix should be real and non-negative because covariance matrices are symmetric and semi positive definite.
However, take a look at the following experiment with ...
- 1,045
24votes
4answers
27kviews
Is there any fast method of matrix exponentiation?
Is there any faster method of matrix exponentiation to calculate Mn (where M is a matrix and n is an integer) than the simple divide and conquer algorithm?
- 2,729
24votes
1answer
27kviews
Identity matrix in Julia
I'm trying to construct the identity matrix in Julia 1.1. After looking at the documentation I found that I could compute a 4x4 Identity matrix as follows:
julia> Id4 =1* Matrix(I, 4, 4)
4×4 ...
- 4,189
24votes
4answers
17kviews
Convolution along one axis only
I have two 2-D arrays with the same first axis dimensions. In python, I would like to convolve the two matrices along the second axis only. I would like to get C below without computing the ...
- 1,660
24votes
3answers
13kviews
Java/Scala library for algebra, mathematics [closed]
Can you advise me some flexible and powerful, yet fast library which could cover SciPy (both in performance and functionality). I found SciPy very expressive - but I want to try something in Scala.
...
- 7,250
24votes
3answers
11kviews
BLAS: gemm vs. gemv
Why does BLAS have a gemm function for matrix-matrix multiplication and a separate gemv function for matrix-vector multiplication? Isn't matrix-vector multiplication just a special case of matrix-...
- 65.7k
24votes
1answer
2kviews
Implementing faster python inner product with BLAS
I found this useful tutorial on using low-level BLAS functions (implemented in Cython) to get big speed improvements over standard numpy linear algebra routines in python. Now, I've successfully ...
- 2,784
23votes
2answers
19kviews
Link ATLAS/MKL to an installed Numpy
TL;DR how to link ATLAS/MKL to existing Numpy without rebuilding.
I have used Numpy to calculate with the large matrix and I found that it is very slow because Numpy only use 1 core to do calculation....
- 633
23votes
4answers
648views
gcc -O0 outperforming -O3 on matrix sizes that are powers of 2 (matrix transpositions)
(For testing purposes) I have written a simple Method to calculate the transpose of a nxn Matrix
void transpose(const size_t _n, double* _A) {
for(uint i=0; i < _n; ++i) {
for(uint j=i+...
- 3,179
22votes
7answers
11kviews
Algorithm for finding nearby points?
Given a set of several million points with x,y coordinates, what is the algorithm of choice for quickly finding the top 1000 nearest points from a location? "Quickly" here means about 100ms on a home ...
- 16.9k
22votes
7answers
19kviews
Vectorizing a gradient descent algorithm
I am coding gradient descent in matlab.
For two features, I get for the update step:
temp0 = theta(1,1) - (alpha/m)*sum((X*theta-y).*X(:,1));
temp1 = theta(2,1) - (alpha/m)*sum((X*theta-y).*X(:,2));
...
- 2,002