Fixing 'Not A Simple Fast-Forward' Error In Backstage Scaffolder

by Lucia Rojas 65 views

Issue Overview

I encountered an issue while using Backstage scaffolder templates. When scaffolding a template from a component, I received the error: Push rejected because it was not a simple fast-forward. Use "force: true" to override. {data={"reason":"not-fast-forward"}}. My goal is to scaffold templates and save the results into a single, unified repository, each in its own unique directory. This approach is crucial because managing numerous APIs across different repositories is not feasible for my use case. To address this, I configured the scaffolder to fetch templates from an origin repository and create the templated files in a specified directory within the unified destination repository.

Problem Description

The primary issue is the not-fast-forward error when pushing changes to the destination repository. I expect that scaffolding a template should create a new directory with a catalog-info.yaml file in the destination repository. However, instead of the expected outcome, the process fails with the aforementioned error. It's worth noting that the destination repository is initially empty, which makes the error perplexing, as there should be no conflicts preventing a fast-forward push.

Expected Behavior

The expected behavior is that the scaffolder should successfully create a new directory (e.g., /sap-cpi-integrations/) in the destination repository, containing the scaffolded files, including catalog-info.yaml. This should occur without any not-fast-forward errors, especially since the destination repository is initially empty.

Reproduction Steps

To reproduce this issue, you can use the following steps and configurations:

Backstage CLI Output

The Backstage CLI environment details are as follows:

OS:   Linux 6.8.0-1031-aws - linux/x64
node: v20.18.1
yarn: 4.4.1
cli:  0.29.6 (installed)
backstage:  1.35.1

Dependencies:
  @backstage/app-defaults                                          1.5.16
  @backstage/backend-app-api                                       1.1.1
  @backstage/backend-common                                        0.25.0
  @backstage/backend-defaults                                        0.7.0
  @backstage/backend-dev-utils                                     0.1.5
  @backstage/backend-openapi-utils                                 0.4.1, 0.5.1
  @backstage/backend-plugin-api                                    1.1.1, 1.2.1
  @backstage/catalog-client                                        1.9.1
  @backstage/catalog-model                                         1.7.3
  @backstage/cli-common                                            0.1.15
  @backstage/cli-node                                              0.2.12
  @backstage/cli                                                   0.29.6
  @backstage/config-loader                                         1.9.5
  @backstage/config                                                1.3.2
  @backstage/core-app-api                                          1.15.4
  @backstage/core-compat-api                                       0.3.5
  @backstage/core-components                                       0.16.3
  @backstage/core-plugin-api                                       1.10.3
  @backstage/e2e-test-utils                                        0.1.1
  @backstage/errors                                                1.2.7
  @backstage/eslint-plugin                                         0.1.10
  @backstage/frontend-app-api                                      0.10.4
  @backstage/frontend-defaults                                     0.1.5
  @backstage/frontend-plugin-api                                   0.9.4
  @backstage/frontend-test-utils                                   0.2.5
  @backstage/integration-aws-node                                  0.1.15
  @backstage/integration-react                                     1.2.3
  @backstage/integration                                           1.16.1, 1.16.                2
  @backstage/plugin-api-docs                                       0.12.3
  @backstage/plugin-app-backend                                    0.4.4
  @backstage/plugin-app-node                                       0.1.29
  @backstage/plugin-app                                            0.1.5
  @backstage/plugin-auth-backend-module-atlassian-provider         0.3.4
  @backstage/plugin-auth-backend-module-auth0-provider             0.1.4
  @backstage/plugin-auth-backend-module-aws-alb-provider           0.3.2
  @backstage/plugin-auth-backend-module-azure-easyauth-provider    0.2.4
  @backstage/plugin-auth-backend-module-bitbucket-provider         0.2.4
  @backstage/plugin-auth-backend-module-bitbucket-server-provider  0.1.4
  @backstage/plugin-auth-backend-module-cloudflare-access-provider 0.3.4
  @backstage/plugin-auth-backend-module-gcp-iap-provider           0.3.4
  @backstage/plugin-auth-backend-module-github-provider            0.2.4, 0.3.1
  @backstage/plugin-auth-backend-module-gitlab-provider            0.2.4
  @backstage/plugin-auth-backend-module-google-provider            0.2.4
  @backstage/plugin-auth-backend-module-guest-provider             0.2.4
  @backstage/plugin-auth-backend-module-microsoft-provider         0.2.4
  @backstage/plugin-auth-backend-module-oauth2-provider            0.3.4
  @backstage/plugin-auth-backend-module-oauth2-proxy-provider      0.2.4
  @backstage/plugin-auth-backend-module-oidc-provider              0.3.4
  @backstage/plugin-auth-backend-module-okta-provider              0.1.4
  @backstage/plugin-auth-backend-module-onelogin-provider          0.2.4
  @backstage/plugin-auth-backend                                   0.24.2
  @backstage/plugin-auth-node                                      0.5.6, 0.6.1
  @backstage/plugin-auth-react                                     0.1.11
  @backstage/plugin-bitbucket-cloud-common                         0.2.27
  @backstage/plugin-catalog-backend-module-github-org              0.3.8
  @backstage/plugin-catalog-backend-module-github                  0.7.11
  @backstage/plugin-catalog-backend-module-logs                    0.1.6
  @backstage/plugin-catalog-backend-module-msgraph                 0.6.8
  @backstage/plugin-catalog-backend-module-scaffolder-entity-model 0.2.4
  @backstage/plugin-catalog-backend                                1.30.0, 1.32.                0
  @backstage/plugin-catalog-common                                 1.1.3
  @backstage/plugin-catalog-graph                                  0.4.15
  @backstage/plugin-catalog-import                                 0.12.9
  @backstage/plugin-catalog-node                                   1.15.1, 1.16.                1
  @backstage/plugin-catalog-react                                  1.15.1
  @backstage/plugin-catalog                                        1.26.1
  @backstage/plugin-events-node                                    0.4.7, 0.4.9
  @backstage/plugin-kubernetes-backend                             0.19.2
  @backstage/plugin-kubernetes-common                              0.9.2
  @backstage/plugin-kubernetes-node                                0.2.2
  @backstage/plugin-kubernetes-react                               0.5.3
  @backstage/plugin-kubernetes                                     0.12.3
  @backstage/plugin-org                                            0.6.35
  @backstage/plugin-permission-backend-module-allow-all-policy     0.2.4
  @backstage/plugin-permission-backend                             0.5.53
  @backstage/plugin-permission-common                              0.8.4
  @backstage/plugin-permission-node                                0.8.7, 0.9.0
  @backstage/plugin-permission-react                               0.4.30
  @backstage/plugin-proxy-backend                                  0.5.10
  @backstage/plugin-proxy-node                                     0.1.0
  @backstage/plugin-scaffolder-backend-module-azure                0.2.5
  @backstage/plugin-scaffolder-backend-module-bitbucket-cloud      0.2.5
  @backstage/plugin-scaffolder-backend-module-bitbucket-server     0.2.5
  @backstage/plugin-scaffolder-backend-module-bitbucket            0.3.6
  @backstage/plugin-scaffolder-backend-module-gerrit               0.2.5
  @backstage/plugin-scaffolder-backend-module-gitea                0.2.5
  @backstage/plugin-scaffolder-backend-module-github               0.5.5
  @backstage/plugin-scaffolder-backend-module-gitlab               0.7.1
  @backstage/plugin-scaffolder-backend                             1.29.0
  @backstage/plugin-scaffolder-common                              1.5.9
  @backstage/plugin-scaffolder-node                                0.6.3
  @backstage/plugin-scaffolder-react                               1.14.4
  @backstage/plugin-scaffolder                                     1.27.5
  @backstage/plugin-search-backend-module-catalog                  0.3.0, 0.3.2
  @backstage/plugin-search-backend-module-pg                       0.5.40
  @backstage/plugin-search-backend-module-techdocs                 0.3.5
  @backstage/plugin-search-backend-node                            1.3.7, 1.3.9
  @backstage/plugin-search-backend                                 1.8.1
  @backstage/plugin-search-common                                  1.2.17
  @backstage/plugin-search-react                                   1.8.5
  @backstage/plugin-search                                         1.4.22
  @backstage/plugin-signals-react                                  0.0.9
  @backstage/plugin-techdocs-backend                               1.11.5
  @backstage/plugin-techdocs-common                                0.1.0
  @backstage/plugin-techdocs-module-addons-contrib                 1.1.20
  @backstage/plugin-techdocs-node                                  1.12.16
  @backstage/plugin-techdocs-react                                 1.2.13
  @backstage/plugin-techdocs                                       1.12.2
  @backstage/plugin-user-settings-common                           0.0.1
  @backstage/plugin-user-settings                                  0.8.18
  @backstage/release-manifests                                     0.0.12
  @backstage/test-utils                                            1.7.4
  @backstage/theme                                                 0.6.3
  @backstage/types                                                 1.2.1
  @backstage/version-bridge                                        1.0.10

Scaffolder Template

The template used for scaffolding is defined as follows:

apiVersion: scaffolder.backstage.io/v1beta3
# https://backstage.io/docs/features/software-catalog/descriptor-format#kind-template
kind: Template
metadata:
  name: api-existing
  title: API Existente
  description: Template de exemplo para criar uma API já existente e documenta-la como um componente rastreável
spec:
  owner: GA_GIT_ARQUITETURA
  type: website

  # These parameters are used to generate the input form in the frontend, and are
  # used to gather input data for the execution of the template.
  parameters:
    - title: Preencha campos do repo
      required:
        - name
      properties:
        description:
          title: Nome descritivo da integração
          type: string
          description: Não utilize espaços, em seu lugar, utilize -(traço) ou _(underline).
          ui:autofocus: true
        name:
          title: Nome Técnico da Integração
          type: string
          description: Não utilize espaços, em seu lugar, utilize -(traço) ou _(underline).
          ui:autofocus: true
        relative_path:
          title: Caminho relativo da integração
          type: string
          description: Descreva o objetivo da solução a ser criada
  # These steps are executed in the scaffolder backend, using data that we gathered
  # via the parameters above.
  steps:
    # Each step executes an action, in this case one templates files into the working directory.
    - id: fetch-base
      name: Fetch Base
      action: fetch:template
      input:
        url: https://github.com/dexco-brasil/backstage-model-api-existing
        targetPath: ./${{ parameters.name }}
        values:
          name: ${{ parameters.name }}
          description: ${{ parameters.description }}
          environment: ${{ parameters.environment }}
          site_url: ${{ parameters.site_url }}
          site_url_title: ${{ parameters.site_url_title }}
    - id: push-catalog-entry
      action: github:repo:push
      name: Publish Repo Push
      input:
        repoUrl: 'github.com/?repo=sap-cpi-integrations&owner=dexco-brasil'
        sourcePath: ./${{ parameters.name }}/
        defaultBranch: master
        gitCommitMessage: 'Register ${{parameters.entityName}}'

    # The final step is to register our new component in the catalog.
    - id: register
      name: Register
      action: catalog:register
      input:
        repoContentsUrl: ${{ steps['push-catalog-entry'].output.repoContentsUrl }}
        catalogInfoPath: '/sap-cpi-integrations/${{ parameters.name }}/catalog-info.yaml'

  # Outputs are displayed to the user after a successful execution of the template.
  output:
    links:
      - title: Repository
        url: ${{ steps['publish'].output.remoteUrl }}
      - title: Open in catalog
        icon: catalog
        entityRef: ${{ steps['register'].output.entityRef }}

Steps to Reproduce

  1. Use the provided scaffolder template (api-existing).
  2. Fill in the required parameters, such as name and description.
  3. Run the scaffolding process.
  4. Observe the error in the logs related to the github:repo:push action, specifically the not-fast-forward error.

Additional Context and Possible Solutions

Understanding the Issue

The not-fast-forward error typically occurs when Git detects that the local branch has diverged from the remote branch. This can happen if there are commits on the remote branch that are not present locally. However, in this case, the destination repository is supposed to be empty, so this error is unexpected.

Potential Causes and Solutions

  1. Incorrect Repository URL: Ensure the repoUrl in the github:repo:push action is correct and points to the intended destination repository. A misconfigured URL can lead to the scaffolder attempting to push to the wrong repository, resulting in this error.
  2. Branching Issues: Verify that the defaultBranch specified in the template (master in this case) exists in the destination repository. If the branch does not exist, it can cause issues with the push operation.
  3. Conflicting Git Configuration: There might be underlying Git configurations that are interfering with the push operation. This is less likely since the repository is supposed to be empty, but it's worth checking for any global or local Git configurations that could be affecting the behavior.
  4. Scaffolder Action Configuration: Double-check the configuration of the github:repo:push action itself. Ensure that all required parameters are correctly set and that there are no conflicting settings.
  5. Force Push (Workaround): As the error message suggests, using force: true in the github:repo:push action can override the error. However, this should be used cautiously, as it can overwrite changes in the repository. This is more of a workaround and not a proper solution.

Detailed Analysis and Troubleshooting Steps

To further diagnose the issue, consider the following steps:

  1. Verbose Logging: Enable verbose logging in the Backstage backend to get more detailed information about the scaffolder process and the Git operations.
  2. Manual Git Push: Try manually pushing a file to the destination repository using Git from the command line. This can help identify if the issue is with the scaffolder or with the Git configuration itself.
  3. Inspect Scaffolder Working Directory: Examine the working directory used by the scaffolder to ensure that the files are being created correctly and that there are no unexpected files or directories.
  4. Check GitHub Permissions: Verify that the Backstage application or service account has the necessary permissions to push to the destination repository.

Template Configuration

Let’s dive deeper into the template configuration. The template’s spec section defines the parameters and steps for the scaffolding process. Here’s a breakdown of the key parts:

  • Parameters: The parameters section defines the input fields required from the user. These include name, description, and relative_path. Ensuring these parameters are correctly captured is essential for the subsequent steps.
  • Steps: The steps section outlines the actions performed during scaffolding. Let's examine each step:
    • Fetch Base (fetch-base): This step fetches the base template from the specified URL (https://github.com/dexco-brasil/backstage-model-api-existing) and places it in a directory named after the name parameter.
    • Publish Repo Push (push-catalog-entry): This is where the issue arises. This step attempts to push the scaffolded files to the destination repository (github.com/?repo=sap-cpi-integrations&owner=dexco-brasil). The sourcePath is set to the directory created in the previous step.
    • Register (register): This final step registers the new component in the catalog using the catalog:register action. It uses the output from the push-catalog-entry step to determine the repository contents URL and the path to the catalog-info.yaml file.

Debugging the github:repo:push Action

The primary focus for debugging should be on the github:repo:push action. Here are some points to consider:

  • Repo URL Format: The repoUrl format in the template (github.com/?repo=sap-cpi-integrations&owner=dexco-brasil) is not standard. The correct format should be github.com/<owner>/<repo>. Ensure that the URL is correctly formatted.
  • Source Path: The sourcePath is set to .//. This should point to the directory containing the scaffolded files. Verify that this path is correct relative to the scaffolder's working directory.
  • Default Branch: The defaultBranch is set to master. Ensure that the destination repository either has a master branch or update this to match the existing branch (e.g., main).
  • Git Commit Message: The gitCommitMessage is set to Register ${{parameters.entityName}}. While this is generally fine, ensure that entityName is correctly populated during the scaffolding process.

Suggested Actions and Guidance

  1. Correct the repoUrl format: Update the repoUrl in the template to the correct format: github.com/dexco-brasil/sap-cpi-integrations.
  2. Verify defaultBranch: Ensure the defaultBranch matches the branch name in the destination repository.
  3. Check GitHub Permissions: Confirm that the Backstage service account has write access to the destination repository.
  4. Enable Verbose Logging: Add verbose logging to the scaffolder backend for more detailed output.
  5. Test Manually: Try a manual Git push to the destination repository to rule out any environment-specific issues.

By addressing these points, the not-fast-forward error should be resolved, allowing the scaffolder to successfully create directories and files in the unified destination repository.

Conclusion

The goal is to streamline API documentation within a single repository using Backstage scaffolder templates. The current issue with the not-fast-forward error is a significant roadblock. By carefully examining the template configuration, particularly the github:repo:push action, and following the detailed troubleshooting steps, you can resolve this issue and achieve the desired outcome of unified API documentation management. Remember, the key is to ensure the repository URL is correctly formatted, the branch names match, and the necessary permissions are in place. If you need further assistance, feel free to provide more context or logs, and I'll be happy to help!

Code of Conduct and Willingness to Submit a PR

I have read the Code of Conduct and I am willing to submit a PR, but I would appreciate some guidance on the best approach.