Fetching Results through SmartUI SDK

---

SmartUI CLI allows you to fetch detailed build results after executing your visual tests. This feature enables you to access comprehensive information about your build and screenshots in a JSON file, making it easier to integrate with your CI/CD pipelines and automation workflows.

Prerequisites

• Basic understanding of Command Line Interface
• Login to TestMu AI SmartUI with your credentials.
• Ensure you are using @lambdatest/smartui-cli version 4.1.43 or higher
• A properly configured SmartUI CLI project

note

If you face any problems executing tests with SmartUI-CLI versions >= v4.x.x, upgrade your Node.js version to v20.3 or above.

Steps to Use

Step 1: Install SmartUI CLI

If you haven't already installed SmartUI CLI, install it using npm:

Global Installation (Recommended):

npm install -g @lambdatest/smartui-cli

Local Installation:

npm install @lambdatest/smartui-cli

Step 2: Configure your Project Token

Setup your project token shown in the SmartUI app after creating your project.

MacOS/Linux

export PROJECT_TOKEN="123456#1234abcd-****-****-****-************"

Windows - CMD

set PROJECT_TOKEN="123456#1234abcd-****-****-****-************"

PowerShell

$env:PROJECT_TOKEN="123456#1234abcd-****-****-****-************"

Step 3: Execute Tests with Results Fetching

You can fetch build results by adding the --fetch-results flag to your test execution command. Here are different ways to use this feature:

Default Usage

If no filename is specified, results will be stored in results.json:

npx smartui --config .smartui.json exec --fetch-results -- <execution-command>

Example: For a Node.js test script :

npx smartui --config .smartui.json exec --fetch-results -- node test.js

Custom Filename

Specify a custom filename for your results:

npx smartui --config .smartui.json exec --fetch-results custom-results.json -- node test.js

Step 4: Understanding the Results

The fetched results JSON file contains detailed information about your build and screenshots. Here's what you'll find in the results file:

{
  "screenshots": {
    "Screenshot-1": [
      {
        "screenshot_name": "Screenshot-1",
        "captured_image": "image_url",
        "baseline_image": "image_url",
        "compared_image": "image_url",
        "browser_name": "edge",
        "viewport": "1920",
        "mismatch_percentage": 3.3,
        "status": "Changes found"
      },
      {
        "screenshot_name": "Screenshot-1",
        "captured_image": "image_url",
        "baseline_image": "image_url",
        "compared_image": "image_url",
        "browser_name": "firefox",
        "viewport": "1366",
        "mismatch_percentage": 4.74,
        "status": "Changes found"
      },
      {
        "screenshot_name": "Screenshot-1",
        "captured_image": "image_url",
        "baseline_image": "image_url",
        "compared_image": "image_url",
        "browser_name": "chrome",
        "viewport": "1366",
        "mismatch_percentage": 4.64,
        "status": "Changes found"
      },
      {
        "screenshot_name": "Screenshot-1",
        "captured_image": "image_url",
        "baseline_image": "image_url",
        "compared_image": "image_url",
        "browser_name": "chrome",
        "viewport": "1920",
        "mismatch_percentage": 3.3,
        "status": "Changes found"
      },
    ],
    "Screenshot-2": [
      {
        "screenshot_name": "Screenshot-2",
        "captured_image": "image_url",
        "baseline_image": "image_url",
        "compared_image": "image_url",
        "browser_name": "edge",
        "viewport": "1920",
        "mismatch_percentage": 0.0,
        "status": "Approved"
      },
      {
        "screenshot_name": "Screenshot-2",
        "captured_image": "image_url",
        "baseline_image": "image_url",
        "compared_image": "image_url",
        "browser_name": "firefox",
        "viewport": "1366",
        "mismatch_percentage": 4.74,
        "status": "Changes found"
      },
      {
        "screenshot_name": "Screenshot-2",
        "captured_image": "image_url",
        "baseline_image": "image_url",
        "compared_image": "image_url",
        "browser_name": "chrome",
        "viewport": "1366",
        "mismatch_percentage": 4.64,
        "status": "Changes found"
      },
      {
        "screenshot_name": "Screenshot-2",
        "captured_image": "image_url",
        "baseline_image": "image_url",
        "compared_image": "image_url",
        "browser_name": "chrome",
        "viewport": "1920",
        "mismatch_percentage": 3.3,
        "status": "Changes found"
      },
    ]
  },
  "build": {
    "build_id": "b420b7a9-77c6-****-****",
    "baseline": false,
    "build_type": "smartui-cli",
    "build_status_ind": "completed",
    "build_status": "pending-approval",
    "commitId": "2b93***",
    "branch": "main",
    "commitAuthor": "John Doe",
    "commitMessage": "Merge pull request from xyz/main"
  },
  "project": {
    "project_id": "1dfb7712-7f20-446f-***-***",
    "name": "Project-Name",
    "username": "johndoe",
    "project_type": "smartui-cli",
    "projectCategory": "web",
    "platform": "cli"
  }
}

Using SmartUI Reporter Tool

The SmartUI Reporter is a web-based tool that allows you to visualize and analyze your SmartUI test results in a tabular format with comprehensive statistics and export capabilities.

Step 1: Obtain Results JSON File

You can get the results.json file using one of the following methods:

Method 1: Export from CLI (Recommended)

Use the --fetch-results flag when executing your SmartUI tests:

For Capture Command:

npx smartui capture urls.json --config config.json --fetch-results results.json

For Exec Command:

npx smartui --config .smartui.json exec --fetch-results results.json -- <execution-command>

Example with Capture:

npx smartui capture urlTest.json --config config.json --fetch-results results.json

Example with Exec:

npx smartui --config .smartui.json exec --fetch-results results.json -- npm test

If no filename is specified, results will be saved as results.json by default:

npx smartui capture urls.json --config config.json --fetch-results

Method 2: Fetch from SmartUI API

You can also fetch the results JSON file directly from the SmartUI API using the Fetch Build Screenshots endpoint.

API Endpoint:

GET /build/screenshots

Parameters:

• project_id (required): Your SmartUI project ID
• build_id (optional): Specific build ID
• build_name (optional): Build name (if both build_id and build_name are provided, build_id takes priority)

Example API Request:

curl -X GET "https://api.lambdatest.com/smartui/build/screenshots?project_id=YOUR_PROJECT_ID&build_id=YOUR_BUILD_ID" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN"

For detailed API documentation, visit the SmartUI API Swagger Documentation.

Step 2: Upload to SmartUI Reporter

1. Navigate to SmartUI Reporter
2. Click on the upload area or drag and drop your results.json file
3. Wait for the file to be processed and parsed

Step 3: View Results

After uploading, you'll see:

• Tabular View: All screenshots organized in a comprehensive table
• Statistics: Key metrics including:
  • Total screenshots
  • Mismatch percentage
  • Status breakdown (Approved, Changes Found, etc.)
  • Browser and viewport distribution
• Screenshot Links: Direct links to:
  • Baseline images
  • Captured images
  • Comparison/diff images
• Build Information: Project details, build ID, branch, commit information

Step 4: Export Options

The SmartUI Reporter provides two export options:

Export to PDF

• Click the Export to PDF button
• Generates a comprehensive PDF report with all screenshots and statistics
• Perfect for sharing with stakeholders or archiving results

Export to CSV

• Click the Export to CSV button
• Exports all data in CSV format including:
  • Page Name
  • Browser
  • Resolution/Viewport
  • Mismatch Percentage
  • Status
  • Screenshot URLs (Baseline, Captured, Diff)
• Ideal for data analysis in spreadsheet applications

Use Cases

• CI/CD Integration: Automatically generate reports after test execution
• Stakeholder Reporting: Share visual test results with non-technical team members
• Historical Analysis: Track visual changes over time
• Debugging: Quickly identify which screenshots have mismatches
• Documentation: Create PDF reports for project documentation

Example Workflow

# 1. Run SmartUI tests and fetch results
npx smartui capture urlTest.json --config config.json --fetch-results results.json

# 2. Upload results.json to SmartUI Reporter
# Visit https://smartui-reporter.netlify.app/ and upload the file

# 3. View results in tabular format with statistics

# 4. Export to PDF or CSV for sharing

Additional Resources

• Comprehensive Troubleshooting Guide
• CLI Documentation
• CLI Exec Commands
• Baseline Management
• Running Your First Project
• SmartUI API Documentation
• SmartUI Reporter Tool
• SmartUI API Swagger Documentation