Error "Invalid JSON data in request body" when receiving Error Reporting API notifications in Cloud

[LucasWPL]

Hello everyone,

I'm facing an issue with my Cloud Run service, which acts as a webhook receiver for Error Reporting API notifications. I would like to request assistance from the community to resolve this problem.

Problem Details:

• I have configured the Error Reporting API alert to send webhook notifications to my Cloud Run service whenever a new error occurs.
• However, I'm experiencing a situation where approximately 30% to 50% of the requests are resulting in an error in my service.
• The specific error message I'm receiving is as follows: "PHP Notice: Hyperf\HttpMessage\Exception\BadRequestHttpException: Invalid JSON data in request body."
• This suggests that the JSON data in the request is not being properly interpreted by my Cloud Run service.
• I would like to know if it's possible for the requests to be arriving with different encoding types, which could explain this inconsistent behavior. It's important to note that my project only receives requests from the Error Reporting API webhook alert.

Steps Taken So Far:

• Verified the configuration of my Cloud Run service to ensure it is correctly set up to receive JSON requests.
• Analyzed the Cloud Run logs and found the mentioned error.
• Conducted additional testing, but have not yet been able to identify the root cause of this issue.

I would appreciate any insights, suggestions, or solutions from anyone who can help resolve this problem. I'm available to provide further information if needed.

Thank you in advance for your assistance.

Best regards, Pedro Lucas

[Eyamba]

Hello Lucas,

Thanks for reaching out.  Sorry to hear that your setup is acting weird. 

Do you mind sharing a sample of what the payload looks like in the cases where the parsing fails?
Feel free to DM it to me if it does contain sensitive information.

Thanks

[LucasWPL]

Hello,

Good morning! Thank you for responding. So, the issue is that my service is unable to handle this request. It breaks when trying to create an object of type Request with the received information and insert it into the application context. Unfortunately, since I'm using Cloud Run and don't have any proxy, I can't identify what exactly is being sent to my service if it doesn't handle the request correctly. That's why I'm raising the suspicion that the encoding type may be causing my service to fail to handle it properly.

Furthermore, it's important to reiterate that this problem doesn't occur in every request. It's worth noting that more than 50% of the requests are processed successfully.

If there's any other information I can provide to assist you, please let me know.

Thank you.

[lguida]

Facing the very same problem. Any news?

[Eyamba]

Hello Friends,

Apologies for my delayed response. This was due to some personal OOO.
Our team has been brainstorming what could be going on here and without a sample of the JSON request causing the exception it is hard to troubleshoot.

Our next step is to take a closer look at your projects, but that requires your explicit permission to access.
To grant this permission, please use the "Send Feedback" feature in the top right hand corner of your Google Cloud Console to summarize the issue,  explicitly call out Error Reporting as the relevant product and include your project number.

This will give us the initial justification to seek permission to look at your project.

Thank you for continuing to partner with us and once again apologies for the delayed response.

[hwsoderlund]

This sounds exactly the same as the bug I just reported to the issue tracker:

https://issuetracker.google.com/issues/323465177

In my case, I am using a Cloud Function to receive the webhook alert, not Cloud Run, but the root cause sounds the same, which is that the webhook request contains invalid json. Specifically, if an error log contains single quotes, those single quotes will be escaped in the json, which is technically invalid. I suggest you debug print the payload you receive from Error Reporting and check if it contains escaped single quotes, or possibly something else causing the json to be invalid. jsonlint.com can be used to quickly validate a json document.

[hwsoderlund]

Also worth mentioning is that in the issue I link to above there is example json in a screen shot. But I am posting a scrubbed example here anyway. Notice the escaped single quotes in 4 places.

{

"version": "1.0",

"subject": "New error in project-name",

"group_info": {

"project_id": "project-name",

"detail_link": "https://console.cloud.google.com/errors/XXX?project=XXX&time=P30D&utm_source=error-reporting-notific..."

},

"exception_info": {

"type": "Error doing stuff",

"message": "thing \'theThing\' (version 17)otherThing (version 22)validation failures: { \"someField\": { \"someOtherField\": \"required\" }}"

},

"event_info": {

"log_message": "Error doing stuff:thing \'theThing\' (version 17)otherThing (version 22)validation failures: { \"someField\": { \"someOtherField\": \"required\" }}\ngoroutine 543325 [running]:STACKTRACE_REMOVED"

"request_method": "",

"request_url": "",

"referrer": "",

"user_agent": "",

"service": "",

"version": "",

"response_status": "0"

}

}

[hwsoderlund]

https://issuetracker.google.com/issues/323401075

[Eyamba]

Hello Friends,

The fix for this specific issue has fully rolled out to prod.
Thank you for flagging it and for your patience with us as we worked to resolve it.
As always, we really value your feedback.
Feel free to reach out if there is any other issue we should look into.