Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Generate ready-to-use CI/CD pipeline configurations that install and run Newman for automated API testing. Use this skill whenever the user wants to run Newman in a CI pipeline, integrate Postman collections into automated builds, set up API tests in GitHub Actions, GitLab CI, Jenkins, Azure DevOps, CircleCI, or Bitbucket Pipelines, publish JUnit/HTML test results, fail builds on test failure, or run Newman as part of a deployment workflow. Trigger on: "Newman in CI", "run Postman tests in pipel
.claude/skills/bilal140202-newman-cicd-integration/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-01 | ✗→✓ | ▲ Improved | 33% | 0% |
| case-02 | ✗→✓ | ▲ Improved | 93% | 0% |
| case-03 | ✗→✓ | ▲ Improved | 90% | 0% |
| case-07 | ✗→✓ | ▲ Improved | 71% | 0% |
| case-08 | ✗→✓ | ▲ Improved | 88% | 0% |
Generate complete, copy-paste-ready CI/CD pipeline configs that install Newman and run Postman collections as part of automated builds.
Before generating a config, determine:
yamlname: API Tests on: push: branches: [main, develop] pull_request: jobs: api-tests: runs-on: ubuntu-latest steps: - name: Checkout repository uses: actions/checkout@v4 - name: Set up Node.js uses: actions/setup-node@v4 with: node-version: '18' - name: Install Newman run: | npm install -g newman npm install -g newman-reporter-htmlextra - name: Run API tests run: | newman run ./collections/my-api.json \ -e ./environments/staging.json \ -r cli,junit,htmlextra \ --reporter-junit-export ./results/junit.xml \ --reporter-htmlextra-export ./results/report.html \ --reporter-htmlextra-title "API Test Results" env: BASE_URL: ${{ secrets.BASE_URL }} API_KEY: ${{ secrets.API_KEY }} - name: Publish test results uses: dorny/test-reporter@v1 if: always() with: name: Newman API Tests path: results/junit.xml reporter: java-junit - name: Upload HTML report uses: actions/upload-artifact@v4 if: always() with: name: api-test-report path: results/report.html
yamlstages: - test api-tests: stage: test image: node:18-alpine before_script: - npm install -g newman newman-reporter-htmlextra script: - | newman run ./collections/my-api.json \ -e ./environments/staging.json \ --env-var "BASE_URL=$BASE_URL" \ --env-var "API_KEY=$API_KEY" \ -r cli,junit,htmlextra \ --reporter-junit-export results/junit.xml \ --reporter-htmlextra-export results/report.html artifacts: when: always reports: junit: results/junit.xml paths: - results/report.html expire_in: 7 days variables: BASE_URL: $BASE_URL # Set in GitLab CI/CD > Variables API_KEY: $API_KEY
groovypipeline { agent any tools { nodejs 'NodeJS-18' // Configure in Global Tool Configuration } stages { stage('Install Newman') { steps { sh 'npm install -g newman newman-reporter-htmlextra' } } stage('Run API Tests') { steps { sh ''' newman run ./collections/my-api.json \ -e ./environments/staging.json \ -r cli,junit,htmlextra \ --reporter-junit-export results/junit.xml \ --reporter-htmlextra-export results/report.html \ --reporter-htmlextra-title "API Tests - ${BUILD_NUMBER}" ''' } } } post { always { junit 'results/junit.xml' publishHTML([ allowMissing: false, alwaysLinkToLastBuild: true, keepAll: true, reportDir: 'results', reportFiles: 'report.html', reportName: 'Newman API Test Report' ]) } } }
yamltrigger: branches: include: - main pool: vmImage: 'ubuntu-latest' steps: - task: NodeTool@0 inputs: versionSpec: '18.x' displayName: 'Set up Node.js' - script: | npm install -g newman newman-reporter-htmlextra displayName: 'Install Newman' - script: | newman run ./collections/my-api.json \ -e ./environments/staging.json \ --env-var "API_KEY=$(API_KEY)" \ -r cli,junit,htmlextra \ --reporter-junit-export $(System.DefaultWorkingDirectory)/results/junit.xml \ --reporter-htmlextra-export $(System.DefaultWorkingDirectory)/results/report.html displayName: 'Run API Tests' env: API_KEY: $(API_KEY) # Set in Pipeline > Variables - task: PublishTestResults@2 condition: always() inputs: testResultsFormat: 'JUnit' testResultsFiles: 'results/junit.xml' testRunTitle: 'Newman API Tests' - task: PublishBuildArtifacts@1 condition: always() inputs: PathtoPublish: 'results/report.html' ArtifactName: 'api-test-report'
yamlversion: 2.1 jobs: api-tests: docker: - image: cimg/node:18.0 steps: - checkout - run: name: Install Newman command: npm install -g newman newman-reporter-htmlextra - run: name: Run API Tests command: | mkdir -p results newman run ./collections/my-api.json \ -e ./environments/staging.json \ --env-var "API_KEY=$API_KEY" \ -r cli,junit,htmlextra \ --reporter-junit-export results/junit.xml \ --reporter-htmlextra-export results/report.html - store_test_results: path: results - store_artifacts: path: results/report.html workflows: test: jobs: - api-tests
Always inject sensitive values as CI environment variables/secrets:
Settings > Secrets and Variables > ActionsSettings > CI/CD > VariablesManage Jenkins > CredentialsPipelines > VariablesReference in Newman via --env-var "KEY=$SECRET_NAME" or pre-set in the environment file.
/
├── collections/
│ └── my-api.json
├── environments/
│ ├── staging.json
│ └── prod.json
└── results/ ← gitignored, created by NewmanAdd results/ to .gitignore.
if: always() / when: alwaysEnsure test result artifacts are published even when Newman exits with a failure code.
Newman exits with code 1 if any tests fail — this automatically fails the pipeline step. Use --bail if you want to stop on the first failure rather than running all tests.
Once the Newman CICD output is delivered, ask the user:
"Would you like me to generate Postman Test Cases for these commands? (yes/no)"
If the user says yes:
You can install it and re-run.
If the user says no:
| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→pass | 10,779 | 4,276 | -60% | 1 | 1 | 0% | 2,349 | 3,128 | +33% | 0 | 0 | — |
case-02 | fail→pass | 9,325 | 6,861 | -26% | 1 | 1 | 0% | 1,868 | 3,609 | +93% | 0 | 0 | — |
case-03 | fail→pass | 10,810 | 7,817 | -28% | 1 | 1 | 0% | 2,069 | 3,930 | +90% | 0 | 0 | — |
case-04 | pass→pass | 8,123 | 10,180 | +25% | 1 | 1 | 0% | 1,646 | 4,082 | +148% | 0 | 0 | — |
case-05 | pass→pass | 6,949 | 4,586 | -34% | 1 | 1 | 0% | 1,420 | 3,232 | +128% | 0 | 0 | — |
case-06 | pass→pass | 12,344 | 11,707 | -5% | 1 | 1 | 0% | 2,742 | 4,685 | +71% | 0 | 0 | — |
case-07 | fail→pass | 10,213 | 6,146 | -40% | 1 | 1 | 0% | 2,063 | 3,531 | +71% | 0 | 0 | — |
case-08 | fail→pass | 10,822 | 6,514 | -40% | 1 | 1 | 0% | 1,862 | 3,498 | +88% | 0 | 0 | — |
case-09 | fail→pass | 12,038 | 7,399 | -39% | 1 | 1 | 0% | 1,875 | 3,619 | +93% | 0 | 0 | — |
case-10 | pass→pass | 11,759 | 8,393 | -29% | 1 | 1 | 0% | 2,203 | 3,988 | +81% | 0 | 0 | — |
case-11 | pass→pass | 6,012 | 5,945 | -1% | 1 | 1 | 0% | 1,056 | 3,377 | +220% | 0 | 0 | — |
case-12 | fail→pass | 13,446 | 9,834 | -27% | 1 | 1 | 0% | 2,077 | 3,920 | +89% | 0 | 0 | — |
case-13 | pass→pass | 7,519 | 7,043 | -6% | 1 | 1 | 0% | 1,310 | 3,363 | +157% | 0 | 0 | — |
case-14 | fail→pass | 7,566 | 5,158 | -32% | 1 | 1 | 0% | 1,154 | 3,058 | +165% | 0 | 0 | — |
case-15 | pass→pass | 7,346 | 5,948 | -19% | 1 | 1 | 0% | 1,218 | 3,099 | +154% | 0 | 0 | — |
case-16 | fail→pass | 9,614 | 6,151 | -36% | 1 | 1 | 0% | 1,804 | 3,406 | +89% | 0 | 0 | — |
case-17 | fail→pass | 9,772 | 6,329 | -35% | 1 | 1 | 0% | 1,799 | 3,459 | +92% | 0 | 0 | — |
case-18 | pass→pass | 8,903 | 4,431 | -50% | 1 | 1 | 0% | 1,629 | 2,994 | +84% | 0 | 0 | — |
case-19 | fail→pass | 7,670 | 5,541 | -28% | 1 | 1 | 0% | 1,635 | 3,445 | +111% | 0 | 0 | — |
case-20 | fail→pass | 8,322 | 6,543 | -21% | 1 | 1 | 0% | 1,624 | 3,425 | +111% | 0 | 0 | — |
case-21 | fail→pass | 7,106 | 4,274 | -40% | 1 | 1 | 0% | 1,414 | 3,027 | +114% | 0 | 0 | — |
case-22 | fail→pass | 10,590 | 5,245 | -50% | 1 | 1 | 0% | 2,149 | 3,277 | +52% | 0 | 0 | — |
case-23 | fail→fail | 15,573 | 1,826 | -88% | 1 | 1 | 0% | 3,255 | 2,535 | -22% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 23 cases were attempted. The headline lift of +61 percentage points is the difference between those two pass rates over the 23 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.