Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
1.1k views
in Technique[技术] by (71.8m points)

logstash - How to parse a json object in the value

I have record in log file with records in JSON:

{"created":1610314252.6277733,"process":13,"levelname":"INFO","message":"Requesting url: http://smshost:8080/push with method: POST and params {'data': {'message': 'Record changed: go to:XXXX', 'resource_id': '1824908f-efae-474e-aa3e-579cabe67517', 'subscriber': '000000000000', 'service': 'SERVICE'}, 'json': None}","message_id":268562185}

Filebeat send this data to logstash

filebeat.prospectors:
- paths:
   - /opt/apps/mobilizer/*.log
  input_type: log
  json.keys_under_root: true
  json.add_error_key: true
  json.message_key: log

Output data:

{
    "@timestamp":"2021-01-11T09:53:48.062Z",
    "@metadata":{"beat":"filebeat","type":"doc","version":"6.8.13"},
    "levelname":"INFO",
    "source":"/opt/apps/mobilizer/logs/test.json",
    "offset":114,
    "message":"Requesting url: http://smshost:8080/push with method: POST and params {'data': {'message': 'Record changed: go to:XXXX', 'resource_id': '1824908f-efae-474e-aa3e-579cabe67517', 'subscriber': '000000000000', 'service': 'SERVICE'}, 'json': None}",
    "beat":{"name":"skif-HP-240-G7-Notebook-PC","hostname":"skif-HP-240-G7-Notebook-PC","version":"6.8.13"},
    "host":{"name":"skif-HP-240-G7-Notebook-PC"},
    "message_id":268562185,
    "error":{"message":"Key 'log' not found","type":"json"},
    "log":{"file":{"path":"/opt/apps/mobilizer/logs/test.json"}},
    "process":13,"created":1.6103142526277733e+09
}

But I want to parse the "message" and get the dictionary data after "paran". (Either in logstash or filebeat ):

{
    ...
    "message":"Requesting url: http://smshost:8080/push with method: POST and params ",
    "data": {"message": "Record changed: go to:XXXX", "resource_id": "1824908f-efae-474e-aa3e-579cabe67517', "subscriber": "000000000000', "service": "SERVICE"}, 
    "json": "None",
    ...
}

How can I do this? Where is it better to do in logstash or in filebeat?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Reply

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
OGeek|极客中国-欢迎来到极客的世界,一个免费开放的程序员编程交流平台!开放,进步,分享!让技术改变生活,让极客改变未来! Welcome to OGeek Q&A Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...