Description

8004scan Extension for Gas Fee Optimization

Table of Contents

1. Overview
2. Motivation
3. Protocol Specification
4. Security Considerations
5. Gas Cost Analysis
6. Adoption Strategy
7. Reference Implementation
8. FAQ

Overview

What is Data URI Compression?

The Data URI Compression Extension is an optional protocol extension designed by 8004scan to dramatically reduce on-chain gas fees for ERC-8004 agent metadata while maintaining the perfect immutability guarantees of Data URI storage.

Key Innovation: Add a compression layer between JSON serialization and base64 encoding, reducing data size by 60-70%.

Problem Statement

Standard Data URI storage format:

data:application/json;base64,<BASE64_ENCODED_JSON>

Issues:

• Expensive: 3KB metadata costs ~69,000 gas for on-chain storage
• Limited adoption: Only 18% of agents use Data URI due to high gas costs
• Prohibitive for production: Most projects choose mutable IPFS/HTTP instead

Solution

Compressed Data URI format:

data:application/json;enc=<algorithm>;base64,<COMPRESSED_THEN_BASE64_ENCODED_DATA>

Benefits:

• 60-70% gas reduction: ~69,000 → ~37,000 gas for typical 3KB metadata
• Perfect immutability: Data stored on-chain, cannot be changed
• Backward compatible: Old parsers will error clearly, not silently fail
• Simple adoption: Single parameter addition to existing Data URI format

Motivation

Why Compression?

Gas Cost Comparison

Storage Method	Immutability	3KB Metadata Gas Cost	Recommendation
Data URI (Uncompressed)	✅✅✅ Perfect	~69,000 gas	High gas cost
Data URI (Compressed) ✨	✅✅✅ Perfect	~37,000 gas	Recommended
IPFS/Arweave	✅✅ Strong	~40,000-70,000 gas (URI storage)	Good balance
HTTP/HTTPS	❌ None	~40,000-70,000 gas (URI storage)	Development only

Insight: Compression makes Data URI practical for production use, not just high-value scenarios.

Note: Gas estimates based on calldata cost formula: Gas = (size_bytes × 16) + 21,000. Actual costs may vary depending on network conditions and contract implementation.

Real-World Impact

Production Agents (4,725 total)

• 18% use Data URI (high-value agents only)
• 51% use IPFS (good immutability, low cost)
• 22% use HTTP/HTTPS (no immutability, lowest cost)

With compression:

• Data URI becomes cost-competitive with IPFS
• Agents can achieve perfect immutability at reasonable cost
• Expected adoption: 40%+ of production agents

Use Cases

1. Regulatory Compliance Agents

Requirement: Immutable audit trail
Solution: Compressed Data URI ensures metadata cannot be tampered with
Gas Reduction: ~149,000 → ~61,000 gas for 8KB full metadata (~59% reduction)

2. Financial Service Agents

Requirement: Transparency and trust
Solution: On-chain metadata provides cryptographic proof
Gas Reduction: ~69,000 → ~37,000 gas for typical 3KB metadata (~46% reduction)

3. Long-Term Archive Agents

Requirement: Guaranteed availability (no IPFS pinning)
Solution: On-chain storage eliminates external dependencies
Savings: 60-70% gas reduction

Protocol Specification

Formal Specification

Grammar (ABNF)

compressed-data-uri = "data:" media-type compression-params ";base64," base64-data

media-type = "application/json"

compression-params = ";enc=" algorithm [";level=" compression-level]

algorithm = "zstd" / "gzip" / "br" / "lz4"

compression-level = 1*2DIGIT  ; Algorithm-specific (e.g., 1-22 for zstd)

base64-data = <base64-encoded compressed data>

Example

data:application/json;enc=zstd;level=9;base64,KLUv/WD8zQAUfRS4VTYwMDE...

Breakdown:

1. data: - Data URI scheme
2. application/json - MIME type (must be JSON)
3. ;enc=zstd - Compression algorithm (required for compression)
4. ;level=9 - Compression level (optional, algorithm-specific)
5. ;base64 - Base64 encoding flag (required)
6. ,KLUv/... - Compressed then base64-encoded payload

Supported Algorithms

Algorithm	Compression Ratio	Speed	Decompression Speed	Default Level	Recommendation
zstd (Zstandard)	⭐⭐⭐⭐⭐ (70%)	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	3	Best Choice ✨
br (Brotli)	⭐⭐⭐⭐⭐ (72%)	⭐⭐⭐	⭐⭐⭐⭐	11	Highest ratio
gzip (GNU zip)	⭐⭐⭐ (60%)	⭐⭐⭐⭐	⭐⭐⭐⭐	6	Best compatibility
lz4 (LZ4)	⭐⭐ (50%)	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	0	Fastest

Recommendation: Zstd level 9 for optimal balance of compression ratio and speed.

Compression Level Guidelines

Zstd (Recommended)

Level 1-3:  Fast compression, lower ratio (~55-65%)
Level 9:    Balanced (recommended) (~70%)
Level 19:   Maximum compression (~75%), 10x slower
Level 22:   Extreme (rarely needed) (~77%), 100x slower

Recommendation: Level 9 (default in reference implementation)

Brotli

Level 4:    Fast (~65%)
Level 11:   Maximum (recommended) (~72%)

Note: Levels 10-11 provide only marginal gains over level 6 but are much slower.

Gzip

Level 6:    Default (~60%)
Level 9:    Maximum (~62%)

Note: Gzip has minimal ratio improvement at higher levels.

LZ4

Level 0:    Fast mode (default) (~50%)
Level 9:    High compression (~55%)

Note: LZ4 prioritizes speed over ratio.

Processing Pipeline

Encoding (Creating Compressed Data URI)

Agent Metadata (JSON object)
    ↓
1. JSON.stringify() with compact separators (no whitespace)
    ↓
2. UTF-8 encode → bytes
    ↓
3. Compress with algorithm (e.g., zstd level 9)
    ↓
4. Base64 encode
    ↓
5. Prepend header: "data:application/json;enc=zstd;level=9;base64,"
    ↓
Compressed Data URI (ready for on-chain storage)

Decoding (Parsing Compressed Data URI)

Compressed Data URI
    ↓
1. Parse header parameters (detect "enc=zstd")
    ↓
2. Extract base64 payload
    ↓
3. Base64 decode → compressed bytes
    ↓
4. Decompress with detected algorithm (with size limit check)
    ↓
5. UTF-8 decode → JSON string
    ↓
6. JSON.parse()
    ↓
Agent Metadata (JSON object)

Security Considerations

1. Zip Bomb Protection ⚠️

Attack Vector: Adversary uploads small compressed data that expands to enormous size

Example:

Compressed: 10KB
Decompressed: 10GB (1000x expansion)
Result: Memory exhaustion, DoS

Defense: Strict decompression size limits

MAX_DECOMPRESSED_SIZE = 100 * 1024  # 100KB hard limit

def safe_decompress(data, algorithm):
    decompressed = decompress_with_algorithm(data, algorithm)

    if len(decompressed) > MAX_DECOMPRESSED_SIZE:
        raise SecurityError("Decompressed data exceeds 100KB (possible zip bomb)")

    return decompressed

8004scan Implementation:

• 100KB decompression limit enforced
• Decompression in async workers (isolated)
• Security events logged for monitoring

2. Algorithm Whitelist

Attack Vector: Malicious algorithm name causes arbitrary code execution

Defense: Strict whitelist of allowed algorithms

ALLOWED_ALGORITHMS = {"zstd", "gzip", "br", "lz4"}

if algorithm not in ALLOWED_ALGORITHMS:
    raise ValueError(f"Algorithm not allowed: {algorithm}")

8004scan Implementation:

• Only 4 algorithms supported
• Reject all unknown algorithms with clear error
• No dynamic algorithm loading

3. Async Decompression

Problem: CPU-intensive decompression can block API requests

Solution: Process in background workers

# API endpoint (returns immediately)
@router.post("/agents")
async def create_agent(agent_uri: str):
    task_id = parse_agent_metadata.delay(agent_uri)
    return {"status": "processing", "task_id": task_id}

# Celery worker (handles decompression)
@celery_app.task
def parse_agent_metadata(uri: str):
    metadata = parse_compressed_data_uri(uri)  # Safe here
    # Store in database...

8004scan Implementation:

• All URI parsing in Celery workers
• API never blocks on decompression
• Worker isolation prevents DoS on API server

4. JSON Validation

Defense: Always validate decompressed data is valid JSON

try:
    metadata = json.loads(decompressed_data)
except json.JSONDecodeError:
    raise ValueError("Decompressed data is not valid JSON")

if not isinstance(metadata, dict):
    raise ValueError("Metadata must be a JSON object (not array or primitive)")

Gas Cost Analysis

Detailed Gas Cost Breakdown

Estimated Gas Costs (based on calldata pricing):

Data Size	Gas (Uncompressed)	Gas (Compressed @ 60%)	Gas Reduction
1KB	~37,000	-	-
2KB	~53,000	~29,000	~24,000 (45%)
3KB	~69,000	~37,000	~32,000 (46%)
5KB	~101,000	~53,000	~48,000 (48%)
8KB	~149,000	~77,000	~72,000 (48%)
10KB	~181,000	~93,000	~88,000 (49%)

Formula:

Gas = (size_bytes × 16) + 21,000  # 16 gas per calldata byte + base tx cost

Compression ratio assumption: 60% (conservative estimate - zstd typically achieves 65-75%)

⚠️ Important: These are theoretical estimates. Actual gas costs and compression ratios need to be measured with real-world on-chain deployment data.

When to Use Compression

Scenario	Uncompressed Size	Recommendation
Minimal metadata	< 1KB	❌ Don't compress (overhead not worth it)
Typical agent	1-3KB	⚖️ Optional (saves ~24,000-32,000 gas)
Full metadata	3-8KB	✅ Recommended (saves ~32,000-72,000 gas)
Rich metadata	8-20KB	✅✅ Strongly recommended (saves ~72,000-176,000 gas)
Complex agent	> 20KB	✅✅ Essential (saves 176,000+ gas)

Rule of Thumb: If metadata > 3KB, always use compression.

Note: Gas savings are estimates based on 60% compression ratio. Actual savings may vary.

Real-World Cost Examples

Example 1: Minimal Agent (1.5KB)

{
  "type": "...",
  "name": "SimpleAgent",
  "description": "Basic agent with one endpoint",
  "image": "ipfs://...",
  "endpoints": [{ "name": "MCP", "endpoint": "..." }]
}

Format	Size	Gas
Uncompressed	1.5KB	~45,000
Compressed	~0.6KB	~30,000
Gas Reduction	-	~15,000 (33%)

Verdict: Marginal gas savings, compression optional.

Example 2: Typical Agent (3KB)

{
  "type": "...",
  "name": "DataAnalyst Pro",
  "description": "...",
  "image": "...",
  "endpoints": [
    {"name": "MCP", "endpoint": "...", "version": "...", "mcpTools": [...]},
    {"name": "A2A", "endpoint": "...", "version": "...", "a2aSkills": [...]},
    {"name": "agentWallet", "endpoint": "..."}
  ],
  "registrations": [...],
  "supportedTrust": ["reputation", "crypto-economic"]
}

Format	Size	Gas
Uncompressed	3KB	~69,000
Compressed	~1.2KB	~37,000
Gas Reduction	-	~32,000 (46%)

Verdict: ✅ Recommended - good gas savings.

Example 3: Full Metadata (8KB)

{
  "type": "...",
  "name": "Enterprise Agent",
  "description": "...",
  "image": "...",
  "endpoints": [
    {"name": "MCP", "endpoint": "...", "mcpTools": [30 tools], "mcpPrompts": [...]},
    {"name": "A2A", "endpoint": "...", "a2aSkills": [50 skills]},
    {"name": "OASF", "endpoint": "...", "skills": [100 skills], "domains": [...]},
    {"name": "agentWallet", "endpoint": "..."}
  ],
  "registrations": [multiple chains],
  "supportedTrust": ["reputation", "crypto-economic", "tee-attestation"],
  "active": true,
  "x402support": true
}

Format	Size	Gas
Uncompressed	8KB	~149,000
Compressed	~2.5KB	~61,000
Gas Reduction	-	~88,000 (59%)

Verdict: ✅✅ Strongly recommended - major gas savings.

Adoption Strategy

Phase 1: Support (Current)

Status: ✅ Complete

8004scan Capabilities:

• ✅ Parser supports compressed Data URIs
• ✅ Documentation published
• ✅ Reference implementation available
• ✅ Security measures implemented

Developer Actions: None required - existing agents work unchanged

Phase 2: Tooling

Status: 🚧 In Progress

8004scan Will Provide:

• Web-based compression tool (paste JSON → get compressed URI)
• CLI tool for batch compression
• Frontend SDK integration examples
• Gas savings calculator

Developer Actions:

• Use tools to generate compressed URIs
• Test with existing agents

Phase 3: Incentivization

Status: 📅 Planned

8004scan Features:

• 💰 Badge on explorer: "Gas-Optimized" for compressed agents
• 📊 Analytics dashboard: Compression adoption rate
• 🏆 Leaderboard: Most gas-efficient agents
• 💡 UI hints: "Save ~32,000 gas by compressing" on registration

Developer Actions:

• Migrate existing agents to compression
• Monitor gas savings

Phase 4: Standardization

Status: 📅 Future

Community Efforts:

• Propose compression extension to ERC-8004 spec
• Submit EIP if adoption > 30%
• Coordinate with other explorers

Developer Actions:

• Provide feedback on protocol
• Participate in governance

Reference Implementation

The 8004scan reference implementation provides a robust framework for handling compressed metadata. Key features:

• Multi-Algorithm Support: Full support for zstd, gzip, brotli, and lz4 algorithms.
• Browser Compatibility: Gzip and deflate support for frontend applications.
• Security First: Built-in zip bomb protection, algorithm whitelisting, and async processing.

For production deployment, refer to the security considerations and follow the recommended implementation patterns.

FAQ

General

Q: Is this an official ERC-8004 feature?
A: No, this is an optional extension proposed by 8004scan. The official ERC-8004 spec does not specify compression. However, this extension is designed to be forward-compatible with future spec updates.

Q: Will old clients break?
A: Old clients will fail gracefully with a clear error message indicating compression is not supported. They will NOT silently misparse the data.

Q: Can I mix compressed and uncompressed agents?
A: Yes, 8004scan supports both formats seamlessly. You can update agents incrementally.

Technical

Q: Which algorithm should I use?
A: Zstd level 9 for best balance. Brotli level 11 for maximum compression (slightly better ratio, slower).

Q: Can I use custom compression algorithms?
A: No, only zstd, gzip, br, and lz4 are allowed for security reasons.

Q: What happens if I exceed 100KB decompressed?
A: Parse will fail with SecurityError. Split metadata across multiple agents or use IPFS for large data.

Q: Does compression work with IPFS URIs?
A: This protocol is for Data URIs only. IPFS URIs should remain uncompressed (already efficient).

Gas & Cost

Q: Is compression worth it for 1KB metadata?
A: Marginal gas savings (~15,000-20,000 gas reduction). Optional unless you value immutability highly.

Q: Is compression still worth it at low gas prices?
A: Yes. Compression saves ~24,000-88,000 gas per agent regardless of gas price. The percentage savings (45-59%) remains constant.

Q: Can I update metadata after registration?
A: Yes, call setAgentURI(agentId, newURI) on the registry contract. However, this costs additional gas (~45,000 gas). Consider using mutable storage (IPFS/HTTP) if frequent updates are needed.

Security

Q: Is this safe?
A: Yes, with proper implementation. 8004scan enforces:

• 100KB decompression size limit (zip bomb protection)
• Algorithm whitelist (no arbitrary code)
• Async processing (no API blocking)
• JSON validation (data integrity)

Q: What if someone tries a zip bomb?
A: Decompression will fail safely with SecurityError. No system impact.

Q: Can metadata be tampered with?
A: No, Data URI metadata is stored on-chain and cryptographically verified by the blockchain.

Appendix

Algorithm Comparison

Test Metadata (3KB JSON)

{
  "type": "https://eips.ethereum.org/EIPS/eip-8004#registration-v1",
  "name": "Test Agent",
  ...
}

Algorithm	Level	Compressed Size	Ratio	Compression Time	Decompression Time
zstd	9	1.2KB	60%	5ms	1ms
zstd	19	1.1KB	63%	50ms	1ms
brotli	11	1.1KB	63%	120ms	3ms
gzip	6	1.5KB	50%	8ms	2ms
gzip	9	1.4KB	53%	15ms	2ms
lz4	0	1.8KB	40%	2ms	1ms

Conclusion: Zstd level 9 offers best balance (good ratio, fast speed).

Changelog

Version	Date	Changes
0.1	2025-12-20	Initial draft - protocol specification and implementation guide

Contact & Feedback

• Maintainer: 8004scan Development Team

Feedback Channels:

• GitHub Issues: Technical bugs and feature requests (coming soon)
• Discord: Community discussion and support (coming soon)
• Email: maintainers@altresear.ch