python - Arrange the multi-similar data efficiently -

-

the datafile showed here measuring record exported instrument.

i uploaded here, interested can download it.

background

sample record-1 fid1, fid2, front_temperature, laser, laserlow, pressure, mode -925    284 1452    315 143 16653    -28500 -924    281 1462    322 136 16641    -28628 -920    281 1455    311 139 16649    -28756 -923    279 1454    312 139 16636    -28884 ......  sample record-2 fid1, fid2, front_temperature, laser, laserlow, pressure, mode -925    284 1452    315 143 16653    -28500 ...... ......

generally, there several record different samples in order of testing routine. , data record these samples in same format.

my attempt

if there 1 sample in datafile( in *.txt format), can arrange datafile pandas. dataframe, can handle data more analysis process in python.

my code shown here: 

# whole datafile several samples record inside open("record.txt") f:      mylist = f.read().splitlines()   ## record each sample length in 803 lines lines = mylist[0:803]  ### sample_name extract third line sample_name = lines[2]  ### each sample, measure record saved in several aspects,  ### regarded columns here columns  = lines[22].split()  ### generate empty columns saving data record later. df  = {columns[0][:-1]:[],columns[1][:-1]:[],columns[2][:-1]:[],columns[3][:-1]:[],columns[4][:-1]:[],   columns[5][:-1]:[],columns[6][:-1]:[],} #### though dumb method  ## data extracting ### valid data record of sample 1 line 23 in range(0, len(lines[23:]),1):     j in range(0, len(columns),1):         df[columns[j][:-1]].append(lines[23+i].split()[j]) pd.dataframe(df)

the result shows this:

enter image description here

my target

from code above, deal datafile 1 sample. when there several samples represented in record text. couldn't find clue deal efficiently.

here illustration of target. generate dataframe dict saving samples records.

enter image description here

any advice appreciate!

i think looking this:

import pandas pd # whole datafile several samples record inside open("record.txt",'r') f:      mylist = f.read().splitlines()   dataset = [] while true:      try:         ## record each sample length in 803 lines         lines, mylist = mylist[0:803], mylist[803:] #this split list!!         ### sample_name extract third line         sample_name = lines[2]            ### each sample, measure record saved in several aspects,          ### regarded columns here         columns  = lines[22].split()          ### generate empty columns saving data record later.         df  = {columns[0][:-1]:[],columns[1][:-1]:[],columns[2][:-1]:[],columns[3][:-1]:[],columns[4][:-1]:[],                columns[5][:-1]:[],columns[6][:-1]:[],} #### though dumb method          ## data extracting         ### valid data record of sample 1 line 23         in range(0, len(lines[23:]),1):             j in range(0, len(columns),1):                 df[columns[j][:-1]].append(lines[23+i].split()[j])      except indexerror:         break      df = pd.dataframe(df)     dataset.append(df)

now dataset[0] should contain df of sample 1.


Comments

Post a Comment

Popular posts from this blog

magento2 - Magento 2 admin grid add filter to collection -

-
i created grid in admin xml ui components. need filter collection via url parameter , dont know how achieve that. tried inject requestinterface collection, filter didnt work. di.xml <virtualtype name="slidelistingdataprovider" type="magento\framework\view\element\uicomponent\dataprovider\dataprovider"> <arguments> <argument name="collection" xsi:type="object" shared="false">xxx\xxx\model\resourcemodel\grid\slide\collection</argument> <argument name="filterpool" xsi:type="object" shared="false">slidelistingfilterpool</argument> <!-- define new object filters --> </arguments> </virtualtype> <virtualtype name="slidelistingfilterpool" type="magento\framework\view\element\uicomponent\dataprovider\filterpool"> <arguments> <argument name="applier...
Read more

Android volley - avoid multiple requests of the same kind to the server? -

-
i using volley upload list of images server. happens within service. once image uploaded remove list. problem arises when internet connection breaks. when internet connection breaks , comes again thinking of adding images yet uploaded request queue. this can result in more 1 copy of same image saved on server request same image might have gone through before getting rseponse. how can address scenario? i'm not quite sure why you're using service schedule requests volley automatically runs it's request in seperate thread. nevertheless can listen both request results, successful , unsuccessful adding listeners. you can remove image list schedule request using volley. if succeeds can carry on desired, if fails can add again list. public static request getimageuploadrequest(final string image) { response.listener<t> responselistener = new response.listener<t>() { @override public void onresponse(t response...
Read more

Combining PHP Registration and Login into one class with multiple functions in one PHP file -

-
i'm trying combine registration, activation , login scripts php website backend script front end developer can pass variables different forms to. my question whether appropriate approach this. don't want have lot of php files different pieces of application developing. far have written following 2 functions login, register , activate user front end developer can call: <?php /** * created phpstorm. * user: karl * date: 26/07/2016 * time: 02:25 */ class users { function register_user($email, $password, $user_name) { $server_name = "localhost"; $u_name = "root"; $db_password = "root"; $db_name = "betamath_graspe"; //email notification variable $from_address="info@slack.com"; //registration form $msg_reg_user='username taken. please choose different username'; $msg_reg_email='email registered'; $msg_reg_active=...
Read more