Deployment Mix-up — Production Restore & Workflow Best Practices

Overview

An impromptu call between Mark Hope and Karly Oykhman to resolve a deployment incident affecting the Finwell U website rebuild. A staging-to-production push overwrote the newly built Finwell U site because the developer (Jeff) had built directly on the production environment rather than staging. Mark restored production from a 7:53 PM backup and prepared to switch the finwellu.com domain to point to the recovered site.

Attendees: Mark Hope, Karly Oykhman
Client: ^[1]

Incident Summary

What Happened

Karly pushed the staging environment to production as part of a standard deployment step. However, because Jeff had built the new Finwell U site directly on the production environment, the push replaced the new site with the old "Financial Wellness Done Right" site that had been sitting on staging. Client Peter was confused to find the old site where the new one should have been.

Root Cause

Jeff built the new site on production rather than staging. He appears to have placed the old site on staging as a reference copy — a workaround that directly caused the overwrite when the staging-to-production push was executed.

Impact

• New Finwell U site was overwritten and temporarily inaccessible at finwellu.com
• Client (Peter) could not log in (was using the old domain URL)
• SSL certificates were disrupted during the restore process

Resolution

Steps Taken

1. Production restore: Mark restored the production environment to the 7:53 PM daily checkpoint backup, recovering the new Finwell U site.
2. Verification: Karly confirmed the restored site at nextlevel-prd1.wpenginepowered.com was the correct new build (identifiable by orange design elements).
3. Domain switch (pending): Mark to set finwellu.com as the primary domain in WP Engine, triggering a URL rewrite across the site.
4. Redirects (pending): Set up redirects from www.finwellu.com and nextlevel-prd1.wpenginepowered.com to finwellu.com to prevent duplicate content indexing by Google.
5. SSL fix (pending): SSL certificates to be reset after domain switch completes.

Client Login Fix

Peter's login issue was caused by attempting to use the old domain. Correct login URL is:

finwellu.com/wp-admin

Credentials remain unchanged.

Key Decisions

• Restore from backup rather than attempt a manual fix — the 7:53 PM checkpoint was clean and recoverable.
• Switch the domain, not push again — once the correct site is on production, only the domain pointer needs to change. No further push is required.
• Redirect all alternate domains to primary — www and WP Engine subdomain variants must redirect to finwellu.com to avoid Google duplicate content penalties.

Action Items

• [ ] Mark: Set finwellu.com as primary domain in WP Engine; fix SSL; set redirects from www.finwellu.com and nextlevel-prd1.wpenginepowered.com → finwellu.com; confirm with Karly when live
• [ ] Karly: Confirm restored site is correct after domain switch
• [ ] Karly: Instruct Peter to use finwellu.com/wp-admin for login
• [ ] Karly: Email Jeff regarding build-on-production practice and correct staging workflow

Best Practices Established

This incident prompted a clear articulation of correct WP Engine deployment workflows. See also: ^[2].

Standard Deployment Workflow

Build on staging → push to production. Never build on production.

• Staging is the construction site. Production is the live building.
• When you push staging → production, WP Engine rewrites all internal URLs from STG to PRD domain variants.
• The primary custom domain (finwellu.com) can be set before or after the push; WP Engine rewrites URLs accordingly.

Keeping an Old Site Live Alongside a New One

If the old site needs to remain publicly accessible while the new one is being built:

1. Spin up a new/separate WP Engine server for the old site
2. Point the old domain (e.g., financialwellnessdoneright.com) to that server
3. Build the new site on the existing server (staging → production workflow)
4. When the new site launches, redirect the old domain to the new one via Cloudflare
5. Decommission the old server when no longer needed

"The old one is just purring along. Do all the building over here. Don't mix apples and oranges." — Mark Hope

Keeping an Old Site for Reference Only (Not Live Traffic)

If the old site only needs to be accessible internally for reference:

1. Copy it to the development environment in WP Engine
2. Add a WP Engine site note (under site settings) clearly stating: "DO NOT PUSH — historical reference only"
3. Communicate explicitly to all team members: do not push the dev environment

⚠️ Risk: The dev environment can still be accidentally pushed to staging, overwriting work. This approach requires clear team communication and is not suitable if the old site needs to serve live traffic.

Why You Can't Run Two Sites on the Same WP Engine Instance

WP Engine environments (production/staging/development) within a single site instance are not independent servers — pushing one overwrites the other. To run two live sites concurrently, they must be on separate server instances with separate domains.

DNS / Domain Concepts (Reference)

• A domain is an address, not the website itself. The website lives on a server.
• Cloudflare manages DNS A records that map domain names to WP Engine server IP addresses.
• WP Engine must also be configured to expect traffic for a given domain — otherwise the server returns a 404 even if DNS is correctly pointed.
• Both sides (DNS in Cloudflare + domain registered in WP Engine) must be configured for a domain to resolve correctly.

Transcript Excerpts

On the root cause:

Mark: "You build in staging, and then when you're done, you send it to production. You don't build in production, and then when you're done, you send it to staging. That's not how you do it."

On the domain vs. site distinction:

Mark: "The URL is not a house. The URL is an address. So there's nothing — that's not the website. The URL is not the website."

On keeping an old site live:

Mark: "If you want to keep the old environment for a while, you create a new server, put that old website on the new server, send traffic to it with the old domain and everything. And then your building over here, it's got nothing to do with the old one."

On the dev environment risk:

Mark: "That development environment thing is kind of fraught with peril too, because somebody can accidentally push the development environment onto the staging site and nuke everything."

On production hygiene:

Mark: "What should be on the production environment? If it's a live website, it should be the live website. If we're building a brand new site, there should be nothing there. It should be empty."