Process pending verifications when the services gets restarted or redeployed

[xmonader]

The service must handle potential downtimes and be capable of recovering pending verifications once it is back online.

[sameh-farouk]

This will be handled in two ways:

1. iDenfy Backend Retries: We’re setting up retries at intervals from the iDenfy dashboard in case our webhook receiver doesn’t return a 200 OK response. This should be sufficient for handling short downtimes (the max retry count is 10, and the max interval is 10 sec, so 100 sec max).

2. Handling Extended Downtime: For extended downtimes that go beyond the retry window, we’ll likely need an alternative approach. One option is to parse a specific email account (configured in the iDenfy dashboard) as a backup when the webhook fails. While the data is available via email, it’s important to note that the email lacks iDenfy's signature, so we would need to retrieve the verification data from the backend to ensure authenticity. This raises the question of whether there’s an alternative way to verify the email sender without directly accessing iDenfy’s servers.

With that said, a more resilient setup (specifically for production environments) should be considered to minimize the chance of any downtime or single point of failure. see here https://github.com/threefoldtech/tf_operations/issues/2897#issuecomment-2462598241