Friday, March 12, 2021

Python Script for Splitting a Delimited File Into Multiple Files

I needed to split a big file delimited by double line breaks into a series of smaller files. Python to the rescue!

Posting this because I didn't find anything on google that put together all the pieces I needed.

def split_file_on_delim(to_split_name, delim, splitted_base, splitted_ext):
    """This function reads a file, splits on delim, and writes each chunk out to its own new file."""   
    fn = open(to_split_name, "r")
    fr = fn.read()
    fs = fr.split(delim)
    for i, chunk in enumerate(fs):
        fw = open(splitted_base + "_%i." %i + splitted_ext,'w')
        fw.write (chunk)
        fw.close() 
#sample usage:  
split_file_on_delim("c:\\tmp\\to_split.txt", "\n\n", "c:\\tmp\\splitted", "txt")