I'm parsing log files where each line is a json string. Here's two example, one works with json.loads (python 2.7 or python 3.7) and the other failed with a delimiter error. The only difference is the values of keys: "cache_response_code" and "server_response"
If the value is 000, it got the error, if it's a normal HTTP response, like "404" it works no problem. If the value is set to 0, it also works. It only fail if the value is 00 or 000 or any multiple 0 value.
Why ?
Example:
import json line1 = '{"localtimestamp": "18/Mar/2020:13:39:04 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_INVALID_URL", "cache_response_code":404, "request_page":"", "server_response":404, "bytes":297, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 0, "ssl_connection": 0, "referrer": "-"}' line_json1 = json.loads(line1) line2 = '{"localtimestamp": "18/Mar/2020:13:39:09 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_UNKNOWN", "cache_response_code":000, "request_page":"/", "server_response":000, "bytes":0, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 5001, "ssl_connection": 0, "referrer": "-"}' line_json2 = json.loads(line2) Output in interpreter:
>>> import json >>> >>> line1 = '{"localtimestamp": "18/Mar/2020:13:39:04 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_INVALID_URL", "cache_response_code":404, "request_page":"", "server_response":404, "bytes":297, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 0, "ssl_connection": 0, "referrer": "-"}' >>> line_json1 = json.loads(line1) >>> >>> line2 = '{"localtimestamp": "18/Mar/2020:13:39:09 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_UNKNOWN", "cache_response_code":000, "request_page":"/", "server_response":000, "bytes":0, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 5001, "ssl_connection": 0, "referrer": "-"}' >>> line_json2 = json.loads(line2) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python2.7/json/__init__.py", line 338, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 366, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode obj, end = self.scan_once(s, idx) ValueError: Expecting , delimiter: line 1 column 134 (char 133) 3 Answers
This seems to be working fine. I have just put the responses codes in double-quotes,
import json line1 = '{"localtimestamp": "18/Mar/2020:13:39:04 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_INVALID_URL", "cache_response_code":"404", "request_page":"", "server_response":"404", "bytes":297, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 0, "ssl_connection":0, "referrer": "-"}' line_json1 = json.loads(line1) line2 = '{"localtimestamp": "18/Mar/2020:13:39:09 -0400", "client_ip":"10.247.69.247", "cache_response":"ERR_UNKNOWN", "cache_response_code":"000", "request_page":"/", "server_response":"000", "bytes":0, "user_agent":"-", "sessionId": "-", "origin_response_time": 0, "client_response_time": 5001, "ssl_connection": 0, "referrer": "-"}' line_json2 = json.loads(line2) Your str is not in good format:
# str like this will work fine s1 = '"{\\"code\\":000}"' json.loads(s1) # works good if do a dumps() before loads() json.loads(json.dumps(line2)) I've found an elegant solution to my problem. instead of json, I use ast
import ast line1 = '{"cache_response_code":404, "referrer": "-"}' line1_dict = ast.literal_eval(line1) line2 = '{"cache_response_code":000, "referrer": "-"}' line2_dict = ast.literal_eval(line2) my real code is:
if files_dict: for file in files_dict.values(): with open(file) as f: lines = f.readlines() for line in lines: line = ast.literal_eval(line) 
