I am using Python 3.8 in Spyder to read a data file with extension ".stg". This files are outputs of some equipments used in Geophysics See file .The only option I have found so far is manually opened the file in a text editor (Notepad), save it as .txt, and finally using this line of code: import pandas as pd df = pd.read_csv('TSC16.txt', header=3) The file opens fine, but I want to know if there is an efficient way to do it because, in the end, my goal is to work with tons of this .stg files. Thank you for you help!

3

1 Answer

The STG file is a plaintext file that has three rows of metadata and then a comma-separated values table. You can read this with pd.read_csv as you have. You will need to set the column names, which you can find here. I have included them in the code below.

import pandas as pd columns = [ 'data record number', 'USER', 'date (YYYYMMDD)', 'time (hh:mm:ss)', 'V/I', '% error in tenths of percent', 'output current in mA', 'apparent resistivity in Ωm or Ωft', 'command file identifier', 'X-coordinate for the A-electrode', 'Y-coordinate for the A-electrode', 'Z-coordinate for the A-electrode', 'X-coordinate for the B-electrode', 'Y-coordinate for the B-electrode', 'Z-coordinate for the B-electrode', 'X-coordinate for the M-electrode', 'Y-coordinate for the M-electrode', 'Z-coordinate for the M-electrode', 'X-coordinate for the N-electrode', 'Y-coordinate for the N-electrode', 'Z-coordinate for the N-electrode', 'Cmd line number', 'Transmitter volt code', '# of measurement cycles', 'Measurement time used', 'Gain setting', 'Channel used', ] df = pd.read_csv("TSC16.stg", skiprows=3, header=None, names=columns)

The dataframe will have to be processed a bit. Two columns have trailing whitespace, and six columns include prefixes like cmd=. The code below takes care of both of these things.

# Strip whitespace from some columns. for colname in ["USER", "command file identifier"]: df.loc[:, colname] = df.loc[:, colname].str.strip() # Remove prefixes like 'cmd=' and convert to a numeric type. for colname in ["Cmd line number", "Transmitter volt code", "# of measurement cycles", "Measurement time used", "Gain setting", "Channel used"]: df.loc[:, colname] = df.loc[:, colname].str.split("=", expand=True).iloc[:, -1].astype(float)

Here is the first row of the dataframe after this processing:

data record number 1 USER USER date (YYYYMMDD) 20190611 time (hh:mm:ss) 11:09:32 V/I 0.00510117 % error in tenths of percent 0 output current in mA 361 apparent resistivity in Ωm or Ωft 0.153848 command file identifier TSC16 X-coordinate for the A-electrode 1.6 Y-coordinate for the A-electrode 0 Z-coordinate for the A-electrode 0 X-coordinate for the B-electrode 0 Y-coordinate for the B-electrode 0 Z-coordinate for the B-electrode 0 X-coordinate for the M-electrode 3.2 Y-coordinate for the M-electrode 0 Z-coordinate for the M-electrode 0 X-coordinate for the N-electrode 4.8 Y-coordinate for the N-electrode 0 Z-coordinate for the N-electrode 0 Cmd line number 1 Transmitter volt code 32 # of measurement cycles 1 Measurement time used 14.4 Gain setting 200 Channel used 1 Name: 0, dtype: object

Note that changing the file extension to .txt does not alter the contents of the file. You can open the STG file in any text editor to see its contents. Here are the first few lines of the file for clarity.

Advanced Geosciences, Inc. SuperSting R8-IP Resistivity meter. S/N: SS1601074 Type: 3D Firmware version: 01.33.74E Survey period: 20190611 Records: 2072 Unit: meter 1,USER ,20190611,11:09:32, 5.10117E-03, 0,361, 1.53848E-01,TSC16 , 1.60000E+00, 0.00000E+00, 0.00000E+00, 0.00000E+00, 0.00000E+00, 0.00000E+00, 3.20000E+00, 0.00000E+00, 0.00000E+00, 4.80000E+00, 0.00000E+00, 0.00000E+00,Cmd=1,HV=32,Cyk=1,MTime=14.4,Gain=200,Ch=1 2,USER ,20190611,11:09:32, 1.39790E-03, 0,361, 1.68638E-01,TSC16 , 1.60000E+00, 0.00000E+00, 0.00000E+00, 0.00000E+00, 0.00000E+00, 0.00000E+00, 4.80000E+00, 0.00000E+00, 0.00000E+00, 6.40000E+00, 0.00000E+00, 0.00000E+00,Cmd=1,HV=32,Cyk=1,MTime=14.4,Gain=200,Ch=2
0