All Questions

Tagged with
Filter by
Sorted by
Tagged with
-1votes
0answers
10views

Ragged nested array need to be converted to array of float

I have a ragged nested array with class 'numpy.ndarray' which I want to convert it to a string of float which I already try to use np.asfarray to change it. the problem is that, my Jupyter Notebook ...
user avatar
0votes
0answers
8views

Dash Plotly error TypeError: Object of type DataFrame is not JSON serializable

Hello I am working with Dash for making dashboard. Below is my code. I tried to fix the error but not able to fix, Can anyone look into this? on chrome i am getting. Error loading layout I am getting ...
user avatar
  • 23
0votes
0answers
10views

Ragged Nested Numpy Array to Tensor

I have ragged nested arrays of .npy file that I have processed. The error is that it my model failed to convert the NumPy array to a Tensor with the remark 'Unsupported object type list') I have try ...
user avatar
0votes
0answers
7views

How to crate a Hybrid Work Model roster simulation

My office is currently manually creating Hybrid work model Rosters. Can we automate the process in Python/Pandas? Rules for creating a roster is: for 2 weeks (10 working days) people should be in ...
user avatar
0votes
1answer
41views

Pandas get rows with subtracting rows as a index

I have a function in python that gets the two rows for a yfinance dataframe of tickers. I am doing a calculation, where I make a row that holds the value obtained from subtracting the value, then ...
user avatar
  • 415
1vote
2answers
15views

the ways of using zip function to generate the cell location based on the row and column information

I created the following dataframe, and would like to identify the cells which are Null, import pandas as pd import numpy as np data = [{'a': 1, 'b': 2, 'c':3}, {'a':10, 'b': np.NaN, 'c':"&...
user avatar
  • 4,679
0votes
1answer
18views

How to change float to date type in python? (ValueError: day is out of range for month)

I have the following column: 0 3012022.0 1 3012022.0 2 3012022.0 3 3012022.0 4 3012022.0 ... 351 24032022.0 352 24032022.0 df.Data = df.Data.astype('...
user avatar
1vote
3answers
37views

Formatting Multiple Columns by Name using .loc

My goal is to state a list of columns by name that I want to apply the formatting to. The section that is commented out is ideally what I am after (which would mean removing the line of code directly ...
user avatar
0votes
2answers
28views

String search on dataframe using key/value from dict

I am trying to match the string present in the below dataframe's 'Disease' column with the key from the dict and if the string is present then change the value in the 'category' column to the value of ...
user avatar
  • 1,122
0votes
1answer
19views

How to use read_excel inside a loop?

when I use read_excel outside a loop it creates a normal dataframe: pd.read_excel('file.xlsx') but I need to parse files in order to create dataframes, like: os.chdir('/path/to/dir') for file in glob....
user avatar
0votes
1answer
17views

In dataframes created by a loop, add suffix to columns that increases with iteration

I am creating multiple dataframes numbered from 1 to n through a loop. The first thing I do for this, is create a dictionary: dict_of_df = {} then runs the loop: for i in range(1, n+1): ... ...
user avatar
  • 702
0votes
0answers
31views

Python Pandas : How to read huge number of csv files from S3 bucket efficiently

I have 750,000 csv input files in S3 bucket which needs to be transformed and loaded into output folder which will also be stored in csv. While I have a notebook which reads these many files , while ...
user avatar
  • 11
0votes
4answers
55views

Dataframe: create pairwise difference

I have this dataframe: Name Score 1 Score 2 Score 3 ... Score 20 Max 2 3 5 ... 2 Jacob 1 4 3 ... 6 How do I get the pairwise difference? So the result should look like: Name Score 1-2 Score 1-3 ...
user avatar
0votes
0answers
28views

How to append dictionaries to a list in Python?

I have create a dictionary named item_dict. After iterating the rows of a data frame, where multiple rows have the same id value, I want to append to item_dict a list ,and to that list to add more ...
user avatar
  • 2,320
0votes
1answer
28views

pandas aggregate column doesnt exist?

Currently i have a dataframe that i am preforming a group by on with aggregate functions. these are the functions aggregation_functions = { '12_months': 'sum', '...
user avatar
0votes
0answers
15views

Selecting data with boolean

I have this data: Time Filename 60Ni 61Ni 62Ni 63Cu 64Ni 65Cu 66Zn 0 input/25.03.2022/220310001_Blk.TXT 0.004213561117649 0.0004941140553758 0.0008264054505464 0....
user avatar
  • 25
0votes
0answers
11views

is there a method to verify existence of column in another pandas df? [duplicate]

i have a dataframe like : Name; address ; firstname ; lastname ; postal_city ; number Martin Lustig; Paris France; Martin; Fabien ; Paris ; 0 Adrian Dubois; Lille France; Adrian; Dubois; Nantes ; 0 ...
user avatar
0votes
1answer
28views

MASK csv columns using python3

I have below requirement to MASK 3 columns (MAILID,SSN,CCNO) with X (each char with X) using python3, I am trying with lambda function, but not getting exact output. any help on this? ID NAME ...
user avatar
  • 453
1vote
1answer
22views

if condition not meet, leave blank python code

how should we write the code that tell python to leave empty cell in dataframe when the condition is not meet? I tries " " like excel but it does not work. I tried 'space' also not work ...
user avatar
0votes
0answers
13views

Make dataframe with user input and even if user directly press "Enter" without input consider no condition check for particulr column

I'm trying to make a data frame considering user input if user press "Enter" without entering anything. It should not check for condition in particular column as well user can input one or ...
user avatar
  • 11
0votes
0answers
27views

Pandas cross merge while combining a number of columns (Python)

I am working in python's pandas with 4 dataframes, each of which have a column of unique identifiers followed by multiple columns of shared attributes. I'd like to generate the cartesian product of ...
user avatar
0votes
2answers
21views

Python Pandas adding a column where the value is across other rows

I would like to use multiple rows in the evaluation of a new column on a pandas dataframe. For a little background lets say that I have rows where data has been aggregated and I need to now how my ...
user avatar
1vote
1answer
28views

Assign count to unique values in in another column by group values in another column [duplicate]

I have a dataframe that is like this Group | People -------------- 1 Cindy 1 Dylan 2 Kathy 3 Steven 3 Jonathan 3 Tiffany And I want to add a new column that adds ...
user avatar
  • 387
0votes
1answer
27views

BMI calculation from two columns of a pandas data frame with missing values

I am still at the beginning of my Python career and I am trying to add a column with the BMI in a DataFrame, which is calculated from two other columns. However, this does not work with my code yet ...
user avatar
  • 51
0votes
0answers
18views

Seaborn heatmap from dataframe with color bars representing metadata

I have the following pandas dataframes: DF = USA Canada Denmark Japan England Spain Brazil mountain 3 1 9 7 1 1 ...
user avatar
1vote
1answer
12views

Filtering on product frequency and category

I think there is a fairly simple solution to this but I just can't wrap my head around it right now. I have a data frame with several hundred thousand orders. I am trying to find the nsmallest margin ...
user avatar
0votes
3answers
38views

Using column as tiebreaker for maximums in Python

Reposted with clarification. I am working on a dataframe that looks like the following: +-------+----+------+------+ | Value | ID | Date | ID 2 | +-------+----+------+------+ | 1 | 5 | 2012 | 111 ...
user avatar
-2votes
0answers
25views

Convert SAS code to Python showing error in a loop

I am trying to migrate a SAS code to python. my SAS code is given below: %do i =1 %to 8; IF index(_muln_string,"&i.")>1 then _muln_cnt=sum(_muln_cnt,1); %end; ...
user avatar
1vote
2answers
37views

Change field in row if other row has same value in different field

I'm looking for a Python-based solution to a common task I often run into. (I know, there must be cool SQL-magic for this, but... not now.) Given a table, we want to change the field f2 if oid is the ...
user avatar
  • 151
2votes
1answer
21views

Pandas - Spreading different values in a column on many columns

I have the following table: Option 1 Option 2 Option 3 A X 1 B X 1 A Y 1 C X 1 B Y 1 I need to split the values of each option on different columns based on the translation as follows: Option 1 ...
user avatar
1vote
2answers
33views

How transform list of strings in column and split dataframe by same string to have several?

I have a dataframe with a column containing list of strings. id sentence category 0 "I love basketball and dunk to the basket" ['...
user avatar
0votes
0answers
24views

Python list, last element overwrites entire list

I am working on a program that reads logfiles in, in a weird format. I have the following code reading from a number of logfiles (logFiles) It works fine to itterate through them and so on. The ...
user avatar
0votes
1answer
19views

Dropping rows from a df rises a SettingWithCopyWarning error (Pandas, Python) when value doesn't exist (using inplace=T)?

I am trying to drop some rows based on a specific value of a dataframe: dd = {'ae': pd.DataFrame(dict(a=[1,2,4], b=[4,5,6])), 'be': pd.DataFrame(dict(a=[13,21,413], b=[456,54,62]))} def ...
user avatar
  • 2,970
0votes
1answer
16views

Check if dictionaries are equal in df

I have a df, in which a column contains dictionaries: a b c d 0 a1 b1 c1 {0.0: 'a', 1.0: 'b'} 1 a2 b2 c2 NaN 2 a3 b3 c3 {0.0: 'cs', 1.0: 'ef', 2.0: 'efg'} and ...
user avatar
  • 67
0votes
3answers
23views

multi-dimensional dictionary to dataframe python

dict_abc = {'A': [[1, 2, 3], [4, 5, 6], [7, 8, 9]], 'B': [[4, 4, 4], [2, 2, 3],], 'C': [[4, 6, 0]] } I would like to convert this to a dataframe in the form | x | y | z | ...
user avatar
  • 13
0votes
2answers
31views

Remove outliers

I'm working on a housing dataset. Below is my CSV file format: BHK Location Price 1 A 10 1 A 100 2 B 50 3 C 80 4 A 100 1 C 500 In ...
user avatar
0votes
0answers
22views

Changing Data Types of Columns in Pandas Dataframe

This is how the df looks: User-ID ISBN Book-Rating Book-Title Book-Author 1 "276725" "034545104X" "0"\\n NaN NaN 2 "276726&...
user avatar
0votes
0answers
11views

Want to Know alternate Solution of excel2img for linux in python

As we Know excel2img is used to convert excel sheet to image in Windows Only. But i want to convert excel to image on linux server. I already use asposecells (using JVM) but i got half sheet image(i.e ...
user avatar
0votes
0answers
9views

Is there a way to export multiple pandas Dataframes in different sheet names using "to_csv" [duplicate]

is there a way to export different Dataframes in one single csv "prediction_dataset.csv" file , but in different sheets . df1 --> sheet_1 df2 --> sheet_2 .. .. dfn --> sheet_n
user avatar
  • 27
-1votes
2answers
31views

How to create two dataframes from a given dataframe?

Assume I have the following data frame: I want to create two data frames such that for any row if column Actual is equal to column Predicted then the value in both columns goes in one data frame ...
user avatar
0votes
0answers
21views

Segmentation fault while using pandas.read_csv

I have a general question regarding what causes Segmentation fault while trying to read csv using pandas.read_csv(). Right now we are trying to analyse the issue but we do not have visibility of the ...
user avatar
0votes
1answer
16views

How to create a dataframe from extracted hashtags?

I have used below code to extract hashtags from tweets. def find_tags(row_string): tags = [x for x in row_string if x.startswith('#')] return tags df['split'] = df['text'].str.split(' ') ...
user avatar
2votes
3answers
36views

Replacing the first occurrence of a value in each row of a pandas DataFrame

I have a DataFrame like this: col1 col2 col3 col4 5 7 12 9 0 9 9 1 9 9 1 1 10 5 2 9 9 3 0 18 Each row has at least one 9 and for each row, I want to replace the first instance of it with 90. ...
user avatar
  • 26.4k
0votes
1answer
25views

Remove rows with invalid date and time data [duplicate]

below is my sample dataframe. I need to remove the columns that have an invalid date or time. Therefore rows 3,5,6 should be deleted. I need a way to verify if the date & time columns contain ...
user avatar
0votes
2answers
25views

Parsing dates and times from a large string into seperate columns

I am trying to parse the date and time from a string column. This is the original column (all one column): description 4/18/2020 21:05 XXXXXXXXXXXXXXXXXXXXXXXXXXX YYYYYYYYYYYY ZZZZZZZZZZZZZZZZZ my ...
user avatar
0votes
0answers
17views

parallelizing a Python function?

I have a function that submits a search job to a REST API, waits for the API to respond, then downloads 2 sets of JSON data, converts the both JSON's into Pandas dataframes, and returns both ...
user avatar
2votes
1answer
31views

Pandas: pivot comma delimited column into multiple columns

I have the following Pandas DataFrame: import pandas as pd import numpy as np df = pd.DataFrame({'id': [1, 2, 3, 4], 'type': ['a,b,c,d', 'b,d', 'c,e', np.nan]}) I need to split the type column based ...
user avatar
  • 57
0votes
1answer
29views

How to get the column name of a dataframe from values in a numpy array

I have a df with 15 columns: df.columns: 0 class 1 name 2 location 3 income 4 edu_level -- 14 marital_status after some transformations I got an numpy.ndarray with ...
user avatar
  • 327
1vote
2answers
56views

Calculation on percentage of a percentage

I am trying to calculate percent of percent. per, per1, and per1-per are in percentages date per per1 per1-per 1/1/2021 0 0 0 2/1/2021 0 0 0 3/1/2021 2 3 ...
user avatar
0votes
1answer
35views

Correlation matrix improving print view removing duplicates

when trying to improve the print view of correlation matrix def view_corr(df): df = df.unstack() corr_f = df.sort_values(kind="quicksort", ascending=False) corr_f = corr_f.dropna(...
user avatar
  • 405

15 30 50 per page
1
2 3 4 5
550