Support deployments for multiple packages in a monorepo context

[julienfouilhe]

In Github Actions there is the environment configuration at the job level, that can be used to use environment secrets and creates a Github Deployment in that environment.

That's great, but it does not work well with monorepos. We have multiple components in our repo, that each have their own deployment job, that is in a component-specific workflow, and all these deploy jobs are run in parallel.

This results in "inactive" deployments, when it's actually the active deployment of a different component.

We could name our environments component_env but then we would not be able to share environment secrets.
There appears to be a task argument in the Github Deployments API but it does not seem to exist in a Github Actions context, and I'm not even sure it would solve the issue?

[david-engelmann]

This is great documentation on using jobs. There is a flag for needs:, which can be used to set jobs that depend on each other. If the jobs don't depend on each other, you can always setup multiple actions with multiple yml files and utilize the security/secrets flow to pass the env vars. Hope this helps!

[julienfouilhe]

Wrong discussion maybe?
I'm talking about the Deployments that are created when using the environment attribute in a job and your reply doesn't talk about that at all.

[sandstrom]

We have this exact same problem.

Would like to use Github Deployment Status APIs with a monorepo. Ideally, there would be a component key for a deployment.

Right now, there is ref (for a Git tag) and payload with arbitrary metadata. Both can be used as a workaround, e.g. a git tag could be web-b230101 or mobile-b230101 for two different components.

However, a blessed component field for monorepos would be nicer.

https://docs.github.com/en/rest/deployments/deployments?apiVersion=2022-11-28#create-a-deployment

[philomory]

This would also be useful if you're deploying the same component multiple times in a single environment, for example, if you build the component and then deploy it to multiple regions using a matrix job. The different regions aren't really "different environments" (prod-us-west-2 and prod-eu-west-1 are still both "production"), so it'd be nice to be able to distinguish them while maintaining a single "environment".

[tracy-ash]

Dealing with this same problem.
Has anyone come up with a solution they like?

[antonosmond]

I've been exploring using Github deployments and we also have teams using monorepos so multiple apps per repo.
If you use the Github deployment API to set the deployment status, you can set the auto_inactive flag to false and handle marking things inactive yourself.
This solves half the problem i.e. other deployments will no longer be marked as inactive, however the Github UI will still only "label" one as active despite multiple active deployments:

In my case, I'm using the task argument when creating the deployments to differentiate apps e.g. deploy:foo and deploy:bar.
After creating a new deployment, once I create the "success" state, I'm using the List Deployments API to get all the deployments for the given environment and app (task), then I create 'inactive' states for them excluding the one I just marked as "success".
As the deployments I'm marking as inactive are filtered by the task property, deployments for different apps (i.e. deployments with a different task) remain active.

[xSAVIKx]

Hey @antonosmond, thx for sharing your approach. Are you updating the deployments from within your GH actions? E.g. after doing the actual deployment?
Maybe you can share an example workflow. Curious to see how one can utilize your approach.

Thanks a lot 🙂

[antonosmond]

Hey @xSAVIKx I'm not doing this via GHA I'm doing it in golang code in a Kubernetes controller, however there's no reason you couldn't call the APIs from a GHA. The docs have examples of calling the APIs using curl, JavaScript and the GH CLI, or you could script it using whatever language you prefer.
I don't have the time to draft a full example right now but in essence you want something like this:

# Create new deployment and use 'task' param for app name e.g. task=deploy:foo. Returns deployment ID.
POST /repos/{owner}/{repo}/deployments

# Create deployment status for the new deployment ID if desired
POST /repos/{owner}/{repo}/deployments/{new_deployment_id}/statuses

# List ALL deployments for the given app e.g. task=deploy:foo
GET /repos/{owner}/{repo}/deployments

# For all listed deployments

  # If the deployment ID is NOT the new deployment ID, mark it as inactive
  # i.e. create a deployment status of inactive for the given deployment IDs
  POST /repos/{owner}/{repo}/deployments/{old_deployment_id}/statuses

The key to this is leveraging the 'task' param in the APIs. That makes it easy to differentiate apps within the same repo and as the list deployments API can also take the 'task' param, it's easy to get all the old deployments for that app and mark them inactive. It's a bit of a hack to be fair but it gets the job done.

[xSAVIKx]

Got it. The problem is that GH actions create a deployment automatically as soon as I specify the environment on the job. I thought you were able to hack it somehow to make it work together 🙂
Yeah, going with a full manual approach is always an option. Thx for the references

[julienfouilhe]

Yes, this thread is specifically a discussion about Github Actions and their automatic creation of deployments using the environment parameter (I've posted it in the "Actions" category), so this solution doesn't apply much: as you can see in my original post, I'm talking about the task argument that you use not being exposed when using GA

[wyardley]

Similar issue where I'm using a matrix deployment to deploy 3-4 different cloud run jobs from the same "environment"; one of them ends up being "active" and the others "inactive". If I want to keep the deploys happening in parallel, and track the

The called deploy workflow:

jobs:
  deploy:
    name: deploy ${{ inputs.job }} ${{ inputs.environment}}
    runs-on: ubuntu-latest
    environment: ${{ inputs.environment}}

This is called via:

  deploy-foo:
    uses: ./.github/workflows/deploy.yml
    needs: build-deploy
    strategy:
      matrix: 
        job:
          - job1
          - job2
    with: 
      environment: foo
      image: ${{ needs.build-deploy.outputs.tag }}
      job: ${{ matrix.job }}

I guess I could make an "environment" per job, but this doesn't seem ideal. Anyone have a workaround that works for this situation?

[kafkas]

Here is a simple workaround for those of you who just want to prevent "Inactive" status in the Deployments tab: just add a dummy job that relies on other jobs to complete, at at the end of your workflow 😛

jobs:
  deploy-service1-dev:
    uses: ./.github/workflows/_deploy-service1.yml
    with:
      env: dev

  deploy-service2-dev:
    uses: ./.github/workflows/_deploy-service2.yml
    with:
      env: dev

  deploy-service3-dev:
    uses: ./.github/workflows/_deploy-service3.yml
    with:
      env: dev

  # Dummy job that executes after all other jobs have finished
  deploy-complete-dev:
    needs: [deploy-service1-dev, deploy-service2-dev, deploy-service3-dev]
    runs-on: ubuntu-latest
    environment: dev # This ensures that this job will do the final deployment to this environment
    if: always()
    steps:
      - name: Check Deployment Status
        run: |
          if [ "${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}" = "true" ]; then
            echo "One or more deployment jobs failed"
            exit 1
          fi
      - name: Deployment Complete
        run: echo "All services have been successfully deployed to dev environment"

[Hebilicious]

@antonosmond here propose a nice workaround : setting the task field to the project you're deploying with a custom action like this one https://github.com/marketplace/actions/deployment-action

However, it would be great to improve the environment flag within jobs.
I believe something simple to specify the task like this would be a good improvement:

deploy-foo:
 environment: production/foo
	
deploy-bar:
 environment: production/bar

I'm not sure if this is the intended way to use task, so maybe another parameter, like project could be introduced.
Then the UI would need some changes as well.
All and all this would be a neat feature, as monorepo are pretty common these days.