Python:Flexible Programming

This page covers some ways Python programs can be made more flexible by using strings and format specifiers.

Loading Files With A Pattern

If you have several files whose names fit a certain pattern (like data01.txt, data02.txt, etc) you can use Python's format command to build a string and then use np.loadtxt to load the file. For instance, with the above pattern and, say, 8 data files with two columns of numbers each, you could access each file and print the average value of each column with:

# %% Imports
import numpy as np

# %% Loop to load data sets
for k in range(8):
    data = np.loadtxt("data{:02.0f}.dat".format(k + 1))
    col1 = data[:,0].copy()
    col2 = data[:,1].copy()
    print("data{:02.0f}.dat column averages are {:0.2e} and {:0.2e}".format(k+1, col1.mean(), col2.mean()))

Loading Various Files

If you have several files (in similar formats) that you need a script to load, rather than hard-coding each load, you can load the files in a loop. There are two fundamentally different ways to go about this:

• If the file names all have a pattern, you can use a formatted string to "build" the filename and then use that string with the appropriate loading command.
• If the files are all in the same folder, or in a folder where it is easy to hand-code files to exclude from loading, you can use the os module in Python and specifically the os.listdir(PATH) method to get a list of strings with the filenames at PATH.

Here is a Trinket demonstrating the latter; note that Trinket does not have the ability to create folders so main.py and the two data files are all in the same place; the code on likes 12-13 causes the loop to skip past any file names included in the list. If yuo aer able to put all your data files in a subfolder, that might work better - just adjust the path variable with the relative path to that subfolder from where the main script is running.

Also, this program shows two different ways of loading files: with pandas and with numpy. See the Pandas page for more specific information on how to load data using pandas.