mercredi 2 septembre 2020

python - Search a list of strings from a text file in a large .csv file

Apologies, as I am probably making a whole host of errors here, but I am trying to search with a list of strings from a file (justgenes.txt) against a large CSV file and return the lines featuring the strings from the justgenes list.

I've been working largely with BASH, but the code I have takes more than 100GB of memory and crashes:

grep -f justgenes.txt allDandHunique.csv > HPCgenesandbugs.csv

Therefore, I am attempting to do it in python, assuming that it will be more efficient, but I have very little knowledge of it.

I am use this code (which I've grabbed from the web), but getting an empty file at the end:

data = open('allDandHunique.csv')
                
with open('justgenes.txt', "r+") as file1:
    fileline1= file1.readlines()
    for x in data: # <--- Loop through the list to check      
        for line in fileline1: # <--- Loop through each line
            if x in line:
                 print(x)

The justgenes file looks like this:

*1A0N_B
1A1A_A
1A4I_A
1A5Y_A
1ACO_A
1AGN_A
1AGS_A
1AJE_A
1AJJ_A
1AP0_A
1APQ_A*,

whilst the csv looks like this:

*"0403181A:PDB=1BP2,2BPP",
"0403181A:PDB=1BP2,2BPP",,,
"0706243A:PDB=1HOE,2AIT,3AIT,4AIT",
"0706243A:PDB=1HOE,2AIT,3AIT,4AIT",,,
"1309311A:PDB=1EMD,2CMD",
"1309311A:PDB=1EMD,2CMD",,,
"1513188A:PDB=1BBC,1POD",
"1513188A:PDB=1BBC,1POD",,,
0308206A,
0308206A,,,
0308221A,
0308221A,,,
0308230A,
0308230A,,,
*

Any help would be gratefully received.

Aucun commentaire:

Enregistrer un commentaire