defi-arbitrage/docs/settlement/as4/DETAILED_NEXT_STEPS.md

AS4 Settlement - Detailed Next Steps

Date: 2026-01-19
Version: 1.0.0

---

Table of Contents

1. Database Setup
2. Environment Configuration
3. Certificate Setup
4. Database Migration
5. Marketplace Seeding
6. Service Configuration
7. Testing
8. Server Startup
9. API Verification
10. Member Onboarding
11. Production Hardening
12. Monitoring Setup
13. Security Audit
14. Documentation Review

---

1. Database Setup

1.1 Verify Database Connectivity

# Check PostgreSQL is running
psql -h 192.168.11.105 -U dbis_user -d dbis_core -c "SELECT version();"

# Verify connection string in .env
grep DATABASE_URL .env

Expected Output: PostgreSQL version and connection success

If Failed:

• Check PostgreSQL service status
• Verify network connectivity to database server
• Verify credentials in .env file

1.2 Verify Database Schema

# Check current schema version
psql -h 192.168.11.105 -U dbis_user -d dbis_core -c "SELECT * FROM _prisma_migrations ORDER BY finished_at DESC LIMIT 5;"

Action: Note the latest migration version

1.3 Backup Database (Production Only)

# Create backup before migration
pg_dump -h 192.168.11.105 -U dbis_user -d dbis_core > backup_$(date +%Y%m%d_%H%M%S).sql

Action: Store backup in secure location

---

2. Environment Configuration

2.1 Create/Update .env File

Add the following environment variables to dbis_core/.env:

# AS4 Gateway Configuration
AS4_BASE_URL=https://as4.dbis.org
AS4_GATEWAY_PORT=8443
AS4_GATEWAY_HOST=0.0.0.0

# Certificate Paths
AS4_TLS_CERT_PATH=/etc/dbis/certs/as4-tls-cert.pem
AS4_TLS_KEY_PATH=/etc/dbis/certs/as4-tls-key.pem
AS4_SIGNING_CERT_PATH=/etc/dbis/certs/as4-signing-cert.pem
AS4_SIGNING_KEY_PATH=/etc/dbis/certs/as4-signing-key.pem
AS4_ENCRYPTION_CERT_PATH=/etc/dbis/certs/as4-encryption-cert.pem
AS4_ENCRYPTION_KEY_PATH=/etc/dbis/certs/as4-encryption-key.pem

# HSM Configuration (if using HSM)
HSM_ENABLED=true
HSM_PROVIDER=softhsm
HSM_SLOT=0
HSM_PIN=your-hsm-pin
HSM_LIBRARY_PATH=/usr/lib/softhsm/libsofthsm2.so

# Redis Configuration (for nonce tracking)
REDIS_URL=redis://localhost:6379
REDIS_PASSWORD=
AS4_NONCE_TTL=300
AS4_NONCE_CLEANUP_INTERVAL=3600

# ChainID 138 Configuration
CHAIN138_RPC_URL=http://192.168.11.250:8545
CHAIN138_WS_URL=ws://192.168.11.250:8546
CHAIN138_ANCHOR_INTERVAL=3600
CHAIN138_CONTRACT_ADDRESS=0x...

# Compliance Configuration
SANCTIONS_SCREENING_ENABLED=true
SANCTIONS_SCREENING_PROVIDER=internal
AML_CHECKS_ENABLED=true
AML_CHECKS_PROVIDER=internal

# Message Processing
AS4_MESSAGE_TIMEOUT=30000
AS4_MAX_MESSAGE_SIZE=10485760
AS4_RATE_LIMIT_PER_MEMBER=1000
AS4_RATE_LIMIT_WINDOW=3600

# Security
AS4_REPLAY_WINDOW_MINUTES=5
AS4_CERTIFICATE_VALIDATION_STRICT=true
AS4_REQUIRE_MESSAGE_SIGNATURE=true
AS4_REQUIRE_MESSAGE_ENCRYPTION=false

# Logging
AS4_LOG_LEVEL=info
AS4_AUDIT_LOG_ENABLED=true
AS4_PAYLOAD_VAULT_ENABLED=true

2.2 Verify Environment Variables

# Check all AS4 variables are set
cd dbis_core
node -e "require('dotenv').config(); console.log('AS4_BASE_URL:', process.env.AS4_BASE_URL);"

Action: Verify all required variables are set

---

3. Certificate Setup

3.1 Generate TLS Certificate (for DBIS)

# Create certificate directory
sudo mkdir -p /etc/dbis/certs
sudo chmod 700 /etc/dbis/certs

# Generate TLS certificate
openssl req -x509 -newkey rsa:2048 \
  -keyout /etc/dbis/certs/as4-tls-key.pem \
  -out /etc/dbis/certs/as4-tls-cert.pem \
  -days 365 -nodes \
  -subj "/CN=as4.dbis.org/O=DBIS/C=US"

# Set permissions
sudo chmod 600 /etc/dbis/certs/as4-tls-key.pem
sudo chmod 644 /etc/dbis/certs/as4-tls-cert.pem

3.2 Generate Signing Certificate

# Generate signing certificate
openssl req -x509 -newkey rsa:2048 \
  -keyout /etc/dbis/certs/as4-signing-key.pem \
  -out /etc/dbis/certs/as4-signing-cert.pem \
  -days 365 -nodes \
  -subj "/CN=DBIS AS4 Signing/O=DBIS/C=US"

# Set permissions
sudo chmod 600 /etc/dbis/certs/as4-signing-key.pem
sudo chmod 644 /etc/dbis/certs/as4-signing-cert.pem

3.3 Generate Encryption Certificate

# Generate encryption certificate
openssl req -x509 -newkey rsa:2048 \
  -keyout /etc/dbis/certs/as4-encryption-key.pem \
  -out /etc/dbis/certs/as4-encryption-cert.pem \
  -days 365 -nodes \
  -subj "/CN=DBIS AS4 Encryption/O=DBIS/C=US"

# Set permissions
sudo chmod 600 /etc/dbis/certs/as4-encryption-key.pem
sudo chmod 644 /etc/dbis/certs/as4-encryption-cert.pem

3.4 Calculate Certificate Fingerprints

# Calculate TLS fingerprint
openssl x509 -fingerprint -sha256 -noout -in /etc/dbis/certs/as4-tls-cert.pem

# Calculate signing fingerprint
openssl x509 -fingerprint -sha256 -noout -in /etc/dbis/certs/as4-signing-cert.pem

# Calculate encryption fingerprint
openssl x509 -fingerprint -sha256 -noout -in /etc/dbis/certs/as4-encryption-cert.pem

Action: Store fingerprints securely for Member Directory registration

3.5 HSM Setup (Production Only)

# Initialize HSM slot (if using SoftHSM)
softhsm2-util --init-token --slot 0 --label "DBIS-AS4" --pin your-pin --so-pin your-so-pin

# Import certificates to HSM
pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so \
  --slot 0 --pin your-pin \
  --write-object /etc/dbis/certs/as4-signing-cert.pem \
  --type cert --id 01 --label "AS4-Signing"

Action: Configure HSM paths in environment variables

---

4. Database Migration

4.1 Review Migration File

# Review migration SQL
cat prisma/migrations/20260119000000_add_as4_settlement_models/migration.sql

Action: Verify migration SQL is correct

4.2 Run Migration (Development)

cd dbis_core

# Generate Prisma client
npx prisma generate

# Run migration
npx prisma migrate dev --name add_as4_settlement_models

Expected Output:

• Migration applied successfully
• 6 new tables created

4.3 Run Migration (Production)

cd dbis_core

# Generate Prisma client
npx prisma generate

# Deploy migration (no prompt)
npx prisma migrate deploy

Expected Output: Migration applied successfully

4.4 Verify Tables Created

# Check tables exist
psql -h 192.168.11.105 -U dbis_user -d dbis_core -c "
SELECT table_name 
FROM information_schema.tables 
WHERE table_schema = 'public' 
AND table_name LIKE 'as4_%'
ORDER BY table_name;
"

Expected Output: 6 tables listed:

• as4_member
• as4_member_certificate
• as4_settlement_instruction
• as4_advice
• as4_payload_vault
• as4_replay_nonce

4.5 Verify Indexes

# Check indexes
psql -h 192.168.11.105 -U dbis_user -d dbis_core -c "
SELECT indexname, tablename 
FROM pg_indexes 
WHERE tablename LIKE 'as4_%'
ORDER BY tablename, indexname;
"

Action: Verify all indexes are created

---

5. Marketplace Seeding

5.1 Review Seed Script

# Review seed script
cat scripts/seed-as4-settlement-marketplace-offering.ts

Action: Verify offering details are correct

5.2 Run Seed Script

cd dbis_core
npx ts-node scripts/seed-as4-settlement-marketplace-offering.ts

Expected Output:

Seeding AS4 Settlement Marketplace Offering...
AS4 Settlement Marketplace Offering created: AS4-SETTLEMENT-MASTER

5.3 Verify Offering in Database

# Check offering exists
psql -h 192.168.11.105 -U dbis_user -d dbis_core -c "
SELECT offeringId, name, status, capacityTier 
FROM \"IruOffering\" 
WHERE offeringId = 'AS4-SETTLEMENT-MASTER';
"

Expected Output: Offering record with status 'active'

5.4 Verify in Marketplace UI

Action:

1. Access Sankofa Phoenix Marketplace
2. Navigate to offerings
3. Verify "AS4 Settlement Master Service" is visible
4. Verify capacity tier is 1
5. Verify pricing is displayed

---

6. Service Configuration

6.1 Redis Setup (for Nonce Tracking)

# Check Redis is running
redis-cli ping

# If not running, start Redis
sudo systemctl start redis
sudo systemctl enable redis

# Test connection
redis-cli -h localhost -p 6379 ping

Expected Output: PONG

6.2 Configure Redis (if needed)

# Edit Redis config
sudo nano /etc/redis/redis.conf

# Set maxmemory and eviction policy
maxmemory 256mb
maxmemory-policy allkeys-lru

# Restart Redis
sudo systemctl restart redis

6.3 ChainID 138 RPC Verification

# Test RPC connection
curl -X POST http://192.168.11.250:8545 \
  -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","method":"eth_chainId","params":[],"id":1}'

Expected Output: {"jsonrpc":"2.0","id":1,"result":"0x8a"} (138 in hex)

If Failed:

• Verify ChainID 138 node is running
• Check network connectivity
• Verify RPC endpoint URL

---

7. Testing

7.1 Unit Tests

cd dbis_core

# Run all tests
npm test

# Run only AS4 tests
npm test -- as4-settlement.test.ts

# Run with coverage
npm test -- --coverage as4-settlement.test.ts

Expected Output: All tests pass

If Tests Fail:

• Review error messages
• Check database connectivity
• Verify test data setup

7.2 Integration Tests

# Run integration tests
npm test -- --testPathPattern=integration

# Run specific integration test
npm test -- src/__tests__/integration/settlement/as4-settlement.test.ts

Action: Verify all integration tests pass

7.3 Manual API Testing

See Section 9: API Verification for detailed API tests

---

8. Server Startup

8.1 Build Application

cd dbis_core

# Build TypeScript
npm run build

# Verify build succeeded
ls -la dist/

Action: Verify dist/ directory contains compiled files

8.2 Start Development Server

cd dbis_core

# Start dev server
npm run dev

Expected Output:

Server running on port 3000
AS4 Gateway initialized
Member Directory initialized
Settlement Core initialized

8.3 Start Production Server

cd dbis_core

# Start production server
NODE_ENV=production npm start

Action: Verify server starts without errors

8.4 Verify Server Health

# Check health endpoint
curl http://localhost:3000/health

Expected Output:

{
  "status": "healthy",
  "timestamp": "2026-01-19T...",
  "version": "1.0.0",
  "database": "connected",
  "hsm": "available"
}

---

9. API Verification

9.1 Health Check

curl -X GET http://localhost:3000/health

Expected: HTTP 200 with health status

9.2 Register Test Member

curl -X POST http://localhost:3000/api/v1/as4/directory/members \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{
    "memberId": "TEST-MEMBER-001",
    "organizationName": "Test Bank",
    "as4EndpointUrl": "https://test-bank.example.com/as4",
    "tlsCertFingerprint": "AA:BB:CC:DD:EE:FF:00:11:22:33:44:55:66:77:88:99:AA:BB:CC:DD:EE:FF:00:11:22:33:44:55:66:77:88:99",
    "allowedMessageTypes": ["DBIS.SI.202", "DBIS.SI.202COV"],
    "routingGroups": ["DEFAULT"],
    "capacityTier": 3
  }'

Expected: HTTP 201 with member record

9.3 Get Member

curl -X GET http://localhost:3000/api/v1/as4/directory/members/TEST-MEMBER-001 \
  -H "Authorization: Bearer YOUR_TOKEN"

Expected: HTTP 200 with member details

9.4 Submit Test Instruction

curl -X POST http://localhost:3000/api/v1/as4/settlement/instructions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{
    "fromMemberId": "TEST-MEMBER-001",
    "payloadHash": "abc123def456",
    "signatureEvidence": {},
    "as4ReceiptEvidence": {},
    "message": {
      "MessageId": "MSG-TEST-001",
      "BusinessType": "DBIS.SI.202",
      "CreatedAt": "2026-01-19T12:00:00Z",
      "FromMemberId": "TEST-MEMBER-001",
      "ToMemberId": "DBIS",
      "CorrelationId": "CORR-001",
      "ReplayNonce": "nonce-123",
      "SchemaVersion": "1.0",
      "Instr": {
        "InstrId": "INSTR-TEST-001",
        "ValueDate": "2026-01-20",
        "Currency": "USD",
        "Amount": "1000.00",
        "DebtorAccount": "MSA:TEST-MEMBER-001:USD",
        "CreditorAccount": "MSA:TEST-MEMBER-002:USD",
        "Charges": "SHA",
        "PurposeCode": "SETT"
      }
    }
  }'

Expected: HTTP 202 with instruction acceptance

9.5 Get Instruction Status

curl -X GET "http://localhost:3000/api/v1/as4/settlement/instructions/INSTR-TEST-001?fromMemberId=TEST-MEMBER-001" \
  -H "Authorization: Bearer YOUR_TOKEN"

Expected: HTTP 200 with instruction status

9.6 Generate Statement

curl -X GET "http://localhost:3000/api/v1/as4/settlement/statements?memberId=TEST-MEMBER-001&accountId=MSA:TEST-MEMBER-001:USD&startDate=2026-01-01&endDate=2026-01-31" \
  -H "Authorization: Bearer YOUR_TOKEN"

Expected: HTTP 200 with statement data

9.7 Export Audit Trail

curl -X GET "http://localhost:3000/api/v1/as4/settlement/audit/INSTR-TEST-001?fromMemberId=TEST-MEMBER-001" \
  -H "Authorization: Bearer YOUR_TOKEN"

Expected: HTTP 200 with audit trail

---

10. Member Onboarding

10.1 Marketplace Subscription

Action:

1. Member accesses Sankofa Phoenix Marketplace
2. Member submits inquiry for AS4 Settlement offering
3. Member completes qualification process
4. Member subscribes to offering

10.2 Automated Provisioning

Action:

1. Deployment orchestrator detects AS4 Settlement subscription
2. Provisioning service creates member record
3. Member receives credentials and endpoint information

10.3 Certificate Registration

# Member submits certificate via API
curl -X POST http://localhost:3000/api/v1/as4/directory/members/MEMBER-XXX/certificates \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer MEMBER_TOKEN" \
  -d '{
    "certificateType": "TLS",
    "fingerprint": "MEMBER_CERT_FINGERPRINT",
    "certificateData": "-----BEGIN CERTIFICATE-----\n...",
    "validFrom": "2026-01-19T00:00:00Z",
    "validTo": "2027-01-19T23:59:59Z"
  }'

Action: Verify certificate is registered and active

10.4 Test Connectivity

Action:

1. Member tests AS4 endpoint connectivity
2. Member sends test message
3. Verify receipt is received
4. Verify message is processed

10.5 Production Activation

Action:

1. Complete test transactions
2. Verify all compliance checks pass
3. Activate member for production
4. Monitor first production transactions

---

11. Production Hardening

11.1 High Availability Setup

# Deploy multiple AS4 gateway instances
# Configure load balancer
# Set up health checks
# Configure auto-scaling

Action:

• Deploy 3+ gateway instances
• Configure load balancer with health checks
• Set up auto-scaling rules
• Configure session affinity if needed

11.2 Database Replication

# Set up PostgreSQL replication
# Configure read replicas
# Set up failover

Action:

• Configure primary-replica setup
• Test failover procedures
• Monitor replication lag

11.3 Redis Cluster

# Set up Redis cluster
# Configure replication
# Set up failover

Action:

• Deploy Redis cluster (3+ nodes)
• Configure replication
• Test failover
• Monitor cluster health

11.4 Backup Configuration

# Configure automated backups
# Set up backup retention
# Test restore procedures

Action:

• Daily full backups
• Hourly incremental backups
• 30-day retention
• Test restore monthly

11.5 Security Hardening

# Review security configuration
# Enable all security features
# Configure firewall rules
# Set up DDoS protection

Action:

• Enable HSM for production
• Configure strict certificate validation
• Enable message encryption
• Set up firewall rules
• Configure DDoS protection (CloudFlare/AWS Shield)

---

12. Monitoring Setup

12.1 Prometheus Configuration

# prometheus.yml
scrape_configs:
  - job_name: 'as4-settlement'
    static_configs:
      - targets: ['localhost:3000']
    metrics_path: '/api/v1/as4/metrics'

Action: Configure Prometheus to scrape AS4 metrics

12.2 Key Metrics to Monitor

• Message processing latency (P99)
• Instruction success rate
• Failed instruction rate
• Certificate expiration warnings
• System availability
• Database connection pool usage
• Redis connection status
• ChainID 138 anchoring status

12.3 Alerting Rules

# alerts.yml
groups:
  - name: as4_settlement
    rules:
      - alert: AS4HighLatency
        expr: as4_message_latency_p99 > 5
        for: 5m
        
      - alert: AS4HighFailureRate
        expr: rate(as4_instructions_failed[5m]) > 0.01
        
      - alert: AS4CertificateExpiring
        expr: as4_certificate_days_until_expiry < 30

Action: Configure alerting rules in Prometheus

12.4 Log Aggregation

# Configure log aggregation (ELK/Loki)
# Set up log retention
# Configure log parsing

Action:

• Set up ELK stack or Loki
• Configure log shipping
• Set up log retention (7 years for audit logs)
• Configure log parsing for AS4 messages

12.5 Dashboard Creation

Action: Create Grafana dashboards for:

• AS4 Gateway metrics
• Settlement processing metrics
• Member activity metrics
• System health metrics
• Compliance metrics

---

13. Security Audit

13.1 Code Security Review

# Run security scanning
npm audit
npm audit fix

# Run SAST tools
# Configure Snyk, SonarQube, etc.

Action:

• Review all security vulnerabilities
• Fix high/critical issues
• Document accepted risks

13.2 Penetration Testing

Action:

• Engage security team for pen testing
• Test AS4 endpoint security
• Test certificate validation
• Test message signing/encryption
• Test replay protection
• Test rate limiting

13.3 Compliance Review

Action:

• Review compliance with rulebook
• Verify audit trail completeness
• Verify evidence storage
• Review sanctions screening integration
• Review AML/CTF checks

13.4 Access Control Review

Action:

• Review RBAC configuration
• Verify HSM access controls
• Review certificate management access
• Review audit log access

---

14. Documentation Review

14.1 Technical Documentation

Action: Review and update:

• API documentation (Swagger/OpenAPI)
• Message schema documentation
• Integration guides
• Architecture diagrams

14.2 Operational Documentation

Action: Review and update:

• Operational runbooks
• Incident response procedures
• Deployment procedures
• Troubleshooting guides

14.3 User Documentation

Action: Create:

• Member onboarding guide
• API integration guide
• Certificate management guide
• FAQ document

14.4 Compliance Documentation

Action: Review:

• Member rulebook
• PKI/CA model documentation
• Security controls documentation
• Audit procedures

---

15. Performance Testing

15.1 Load Testing

# Use k6, JMeter, or similar
# Test message processing throughput
# Test concurrent member connections
# Test database load

Action:

• Test with 100 concurrent members
• Test 1000 messages/second throughput
• Test P99 latency under load
• Identify bottlenecks

15.2 Stress Testing

Action:

• Test system behavior under extreme load
• Test failover scenarios
• Test recovery procedures
• Document limits and thresholds

15.3 Endurance Testing

Action:

• Run system for 24+ hours
• Monitor memory leaks
• Monitor database growth
• Monitor log file sizes

---

16. Disaster Recovery

16.1 DR Plan Documentation

Action: Document:

• Recovery time objectives (RTO)
• Recovery point objectives (RPO)
• Backup procedures
• Restore procedures
• Failover procedures

16.2 DR Testing

Action:

• Test database restore
• Test service failover
• Test certificate recovery
• Test audit log recovery
• Document test results

---

17. Go-Live Checklist

17.1 Pre-Go-Live

• All tests passing
• Security audit complete
• Performance testing complete
• Documentation complete
• Team training complete
• Monitoring configured
• Alerting configured
• Backup procedures tested
• DR procedures tested

17.2 Go-Live

• Database migration applied
• Marketplace offering active
• Services running
• Monitoring active
• Support team on standby
• Communication sent to stakeholders

17.3 Post-Go-Live

• Monitor first 24 hours closely
• Review all alerts
• Verify all transactions processed
• Collect feedback from members
• Document any issues
• Schedule post-mortem

---

18. Ongoing Operations

18.1 Daily Tasks

• Review health checks
• Review error logs
• Check certificate expiration warnings
• Review member activity
• Monitor system metrics

18.2 Weekly Tasks

• Review performance metrics
• Review security alerts
• Review compliance reports
• Update documentation if needed

18.3 Monthly Tasks

• Review audit logs
• Review member onboarding
• Review system capacity
• Update runbooks
• Security review

18.4 Quarterly Tasks

• Disaster recovery testing
• Security audit
• Performance review
• Documentation review
• Capacity planning

---

19. Troubleshooting Common Issues

19.1 Database Connection Issues

Symptoms: Migration fails, services can't connect

Actions:

1. Check PostgreSQL service status
2. Verify network connectivity
3. Check credentials in .env
4. Verify database exists
5. Check firewall rules

19.2 Certificate Issues

Symptoms: TLS handshake fails, signature validation fails

Actions:

1. Verify certificate paths in .env
2. Check certificate permissions
3. Verify certificate validity
4. Check certificate fingerprints match
5. Review certificate expiration

19.3 Redis Connection Issues

Symptoms: Nonce validation fails, replay protection not working

Actions:

1. Check Redis service status
2. Verify Redis URL in .env
3. Test Redis connectivity
4. Check Redis memory usage
5. Review Redis logs

19.4 Message Processing Failures

Symptoms: Instructions rejected, errors in logs

Actions:

1. Check instruction logs
2. Verify member status
3. Check compliance gates
4. Review liquidity/limits
5. Check posting engine status

---

20. Support and Maintenance

20.1 Support Channels

• Email: [email protected]
• Slack: #as4-settlement
• On-call: PagerDuty rotation
• Documentation: /docs/settlement/as4/

20.2 Maintenance Windows

• Scheduled: Monthly, 2-hour window
• Emergency: As needed
• Notification: 7 days advance notice

20.3 Version Updates

• Process: Follow semantic versioning
• Testing: Test in staging first
• Deployment: Blue-green deployment
• Rollback: Automated rollback on failure

---

Summary Checklist

Immediate (Before Go-Live)

• Database migration applied
• Marketplace offering seeded
• Environment variables configured
• Certificates generated and installed
• Redis configured
• ChainID 138 RPC verified
• All tests passing
• Server starts successfully
• API endpoints verified
• Monitoring configured

Short-term (First Week)

• Member onboarding tested
• Production transactions monitored
• Performance metrics reviewed
• Security audit completed
• Documentation finalized

Long-term (Ongoing)

• Regular security audits
• Performance optimization
• Capacity planning
• Feature enhancements
• Member feedback integration

---

End of Detailed Next Steps