APIs

Operational

blocks.dopt.com

99.90% uptime
Aug 2023 · 100.0%Sep · 100.0%Oct · 99.72%
Aug 2023100.0% uptime
Sep 2023100.0% uptime
Oct 202399.72% uptime

users.dopt.com

99.92% uptime
Aug 2023 · 100.0%Sep · 100.0%Oct · 99.76%
Aug 2023100.0% uptime
Sep 2023100.0% uptime
Oct 202399.76% uptime

ai.dopt.com

100.0% uptime
Aug 2023 · 100.0%Sep · 100.0%Oct · 100.0%
Aug 2023100.0% uptime
Sep 2023100.0% uptime
Oct 2023100.0% uptime

App

Operational

app.dopt.com

99.95% uptime
Aug 2023 · 99.93%Sep · 99.94%Oct · 100.0%
Aug 202399.93% uptime
Sep 202399.94% uptime
Oct 2023100.0% uptime

Notice history

Oct 2023

blocks.dopt.com is reporting degraded performance on transitions and intents
  • Resolved
    Resolved

    This incident has been resolved.

    After monitoring, we determined that a version of the Blocks API which began rolling out at approximately 3:00 PM PDT had an incorrectly configured client for one of our new internal services. This client began throwing errors because it could not reach its internal service due to a wrong internal DNS reference.

    We first rolled back to a previous version of the Blocks API (from this morning at 9:00 AM) and confirmed that the API was functional (no 5xx reported and e2e tests all passing).

    We subsequently rolled out updates to correct the wrong internal DNS reference and also re-configured the client to not crash the API when incorrectly configured.

  • Monitoring
    Monitoring

    We've completed all our rollbacks.

    Additionally, we're beginning to redeploy our earlier performance and logging updates in a sequential manner. We've added further redundancy checks in our code to prevent the Blocks API from crashing on transitions and intents.

  • Identified
    Identified

    We have started rolling back several potentially problematic updates we made this afternoon to the Blocks API to mitigate the degraded performance.

    We are updating the status of this incident to a partial outage until we can fully remedy the 5xx errors.

  • Investigating
    Investigating

    On block transitions and flow intents, Blocks API is reporting degraded performance with intermittent 5xx errors.

Aug 2023

Aug 2023 to Oct 2023