Skip to content

ENTERPRISE.md

Latest commit

 

History

History
730 lines (546 loc) · 15.8 KB

ENTERPRISE.md

File metadata and controls

730 lines (546 loc) · 15.8 KB
Raw

Enterprise Deployment Guide

stout is designed for enterprise environments with features for private hosting, CI/CD integration, air-gapped installations, and compliance requirements.

Table of Contents

Use Cases

Internal Development Teams

  • Standardized tooling: Ensure all developers have the same package versions
  • Reproducible builds: Lock files guarantee consistent environments
  • Fast onboarding: New developers get productive in minutes

CI/CD Pipelines

  • Cached dependencies: Offline mirrors eliminate network latency
  • Deterministic builds: Pinned versions prevent "works on my machine"
  • Parallel execution: Multiple jobs share the same package cache

Regulated Industries

  • Audit trails: Track every package installation
  • Approved packages: Curate allowed packages via private index
  • Vulnerability management: Automated scanning with stout audit

Air-Gapped Environments

  • Offline operation: Full functionality without internet access
  • Controlled updates: Manual mirror synchronization
  • Security compliance: No external network dependencies

Private Index Hosting

Host your own stout index for complete control over available packages.

Option 1: GitHub (Private Repository)

The simplest approach for teams already using GitHub:

# Fork the stout-index repository to your organization
# Make it private
# Update stout configuration

~/.stout/config.toml:

[index]
base_url = "https://raw.githubusercontent.com/YOUR_ORG/stout-index/main"

For authentication, set a GitHub token:

export STOUT_GITHUB_TOKEN="ghp_xxxxx"

Option 2: Static File Server

Host the index on any web server (nginx, Apache, S3, etc.):

# Sync the index locally
cd /path/to/index
./scripts/sync_all.sh

# Structure:
# /path/to/index/
# ├── manifest.json
# ├── index.db.zst
# ├── formulas/
# │   ├── manifest.json
# │   ├── index.db.zst
# │   └── data/
# │       ├── a/
# │       │   └── ack.json.zst
# │       └── ...
# └── casks/
#     └── ...

nginx configuration:

server {
    listen 443 ssl;
    server_name stout.internal.company.com;

    ssl_certificate /etc/ssl/stout.crt;
    ssl_certificate_key /etc/ssl/stout.key;

    root /var/www/stout-index;

    location / {
        autoindex off;
        add_header Cache-Control "public, max-age=300";
    }

    # Enable gzip for JSON files
    gzip on;
    gzip_types application/json;
}

Option 3: S3 / Cloud Storage

For AWS S3:

# Sync index to S3
aws s3 sync /path/to/index s3://company-stout-index/ \
    --content-encoding zstd \
    --cache-control "max-age=300"

# Configure CloudFront for HTTPS

~/.stout/config.toml:

[index]
base_url = "https://stout.company.cloudfront.net"

Curating Packages

Create a custom index with only approved packages:

#!/usr/bin/env python3
"""Sync only approved formulas to private index."""

import json
from pathlib import Path

APPROVED_FORMULAS = [
    "git", "curl", "jq", "yq", "python@3.11", "node@20",
    "go", "rust", "cmake", "make", "gcc",
    # Add your approved packages
]

def filter_index(source_dir: Path, dest_dir: Path):
    """Copy only approved formulas to destination."""
    for formula in APPROVED_FORMULAS:
        first_char = formula[0]
        src = source_dir / "formulas" / "data" / first_char / f"{formula}.json.zst"
        if src.exists():
            dst = dest_dir / "formulas" / "data" / first_char / f"{formula}.json.zst"
            dst.parent.mkdir(parents=True, exist_ok=True)
            shutil.copy(src, dst)

    # Rebuild the SQLite index with only approved formulas
    # (Use the sync scripts with a filter)

Custom Signing Keys

Generate and use your own signing keys for complete trust chain control.

Generate a Keypair

cd /path/to/stout-index/scripts

# Install dependencies
uv sync

# Generate new keypair
uv run python sign_index.py generate --output ../keys

# Output:
# Generated keypair:
#   Private key: ../keys/stout-index.key
#   Public key:  ../keys/stout-index.pub
#
# Public key (hex): abc123...

Configure stout to Trust Your Key

Option 1: Replace the default key (requires building from source)

Edit crates/stout-index/src/signature.rs:

pub const DEFAULT_PUBLIC_KEY_HEX: &str = "YOUR_PUBLIC_KEY_HEX";

Option 2: Add as additional trusted key (runtime configuration)

~/.stout/config.toml:

[security]
additional_trusted_keys = [
    "YOUR_PUBLIC_KEY_HEX"
]

Sign Your Index

# Using key file
uv run python sign_index.py sign \
    --key ./keys/stout-index.key \
    --index-dir /path/to/index

# Using environment variable (for CI)
export STOUT_SIGNING_KEY="private_key_hex"
uv run python sign_index.py sign \
    --key '$STOUT_SIGNING_KEY' \
    --index-dir /path/to/index

Key Management Best Practices

  1. Store private keys securely: Use HashiCorp Vault, AWS Secrets Manager, or similar
  2. Rotate keys periodically: Add new keys to additional_trusted_keys before rotating
  3. Backup keys: Losing the private key requires re-signing all indexes
  4. Audit key usage: Log all signing operations

CI/CD Integration

GitHub Actions

name: Build with stout

on: [push, pull_request]

jobs:
  build:
    runs-on: macos-latest  # or ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Install stout
        run: |
          curl -fsSL https://raw.githubusercontent.com/neul-labs/stout/main/install.sh | bash
          echo "$HOME/.local/bin" >> $GITHUB_PATH

      - name: Cache stout packages
        uses: actions/cache@v4
        with:
          path: |
            ~/.stout/downloads
            ~/.stout/cache
          key: stout-${{ runner.os }}-${{ hashFiles('Brewfile.lock') }}
          restore-keys: |
            stout-${{ runner.os }}-

      - name: Install dependencies
        run: |
          stout update
          stout bundle install

      - name: Build
        run: make build

GitLab CI

stages:
  - setup
  - build

variables:
  STOUT_CACHE_DIR: ${CI_PROJECT_DIR}/.stout-cache

install-deps:
  stage: setup
  image: ubuntu:22.04
  cache:
    key: stout-${CI_COMMIT_REF_SLUG}
    paths:
      - .stout-cache/
  script:
    - curl -fsSL https://raw.githubusercontent.com/neul-labs/stout/main/install.sh | bash
    - export PATH="$HOME/.local/bin:$PATH"
    - stout update
    - stout bundle install
  artifacts:
    paths:
      - /opt/homebrew/Cellar/

build:
  stage: build
  needs: [install-deps]
  script:
    - make build

Jenkins

pipeline {
    agent any

    environment {
        STOUT_CACHE = "${WORKSPACE}/.stout-cache"
    }

    stages {
        stage('Setup') {
            steps {
                sh '''
                    curl -fsSL https://raw.githubusercontent.com/neul-labs/stout/main/install.sh | bash
                    export PATH="$HOME/.local/bin:$PATH"
                    stout update
                    stout bundle install
                '''
            }
        }

        stage('Build') {
            steps {
                sh 'make build'
            }
        }
    }

    post {
        always {
            // Cache cleanup
            sh 'stout cleanup --prune=7'
        }
    }
}

Brewfile for CI

Create a Brewfile for reproducible dependencies:

# Brewfile
# Build tools
brew "cmake"
brew "make"
brew "ninja"

# Languages
brew "python@3.11"
brew "node@20"
brew "go"
brew "rust"

# Utilities
brew "jq"
brew "yq"
brew "curl"
brew "git"

# Testing
brew "shellcheck"
brew "hadolint"

Lock Files

Generate and commit lock files for reproducibility:

# Generate lock file from installed packages
stout lock generate

# Install exact versions from lock file
stout lock install

# Verify lock file matches installed
stout lock verify

Brewfile.lock format:

{
  "generated_at": "2024-11-27T12:00:00Z",
  "stout_version": "0.1.0",
  "packages": {
    "jq": {
      "version": "1.7.1",
      "bottle_sha256": "abc123...",
      "dependencies": ["oniguruma"]
    }
  }
}

Air-Gapped Installations

For environments without internet access.

Create an Offline Mirror

# Create mirror with specific packages
stout mirror create /path/to/mirror jq curl git python@3.11

# Include all dependencies
stout mirror create /path/to/mirror --with-deps jq curl

# Mirror everything in a Brewfile
stout mirror create /path/to/mirror --from-brewfile ./Brewfile

# Include casks
stout mirror create /path/to/mirror --casks firefox visual-studio-code

Transfer to Air-Gapped System

# Create archive
tar -czvf stout-mirror.tar.gz /path/to/mirror

# Transfer via approved method (USB, secure file transfer, etc.)
# On air-gapped system:
tar -xzvf stout-mirror.tar.gz -C /opt/

Configure for Offline Use

~/.stout/config.toml:

[index]
# Point to local mirror
base_url = "file:///opt/stout-mirror"

[security]
# May need to adjust for offline use
allow_unsigned = true  # Or use your own signing key

Serve Mirror on Internal Network

# Start HTTP server
stout mirror serve /opt/stout-mirror --port 8080 --bind 0.0.0.0

# Or use nginx/Apache for production

Update Mirror

# On internet-connected system
stout mirror update /path/to/mirror

# Verify integrity
stout mirror verify /path/to/mirror

# Transfer updated files
rsync -av /path/to/mirror/ user@airgapped:/opt/stout-mirror/

Mirror Security Options

Mirrors support two security models:

Option 1: Preserve Upstream Signatures (Recommended)

The mirror preserves the original official signature:

# Create mirror - upstream signature is automatically preserved
stout mirror create /path/to/mirror jq curl git

# The manifest.json will include:
# {
#   "upstream_signature": {
#     "signature": "original_ed25519_sig...",
#     "index_sha256": "original_hash...",
#     "signed_at": 1732723200
#   }
# }

Clients verify against the official stout public key.

Option 2: Enterprise Re-signing

Sign the mirror with your own key for full control:

# Generate enterprise keypair (one-time)
cd /path/to/signing-tools
uv run python sign_index.py generate --output ./keys
# Save keys/stout-index.key securely (e.g., HashiCorp Vault)
# Distribute keys/stout-index.pub to clients

# Sign the mirror
uv run python sign_index.py sign \
    --key ./keys/stout-index.key \
    --index-dir /path/to/mirror

# Configure clients
# ~/.stout/config.toml
[security]
additional_trusted_keys = ["YOUR_ENTERPRISE_PUBLIC_KEY"]

Comparison:

Aspect Preserve Upstream Enterprise Re-sign
Setup complexity None Requires key management
Trust chain Official → Mirror → Client Enterprise → Client
Key rotation Handled by stout team Self-managed
Custom packages Not supported Fully supported
Audit trail Traceable to official Internal only

Multi-Prefix Environments

Isolate dependencies for different projects or teams.

Project-Specific Prefixes

# Create isolated environment for project
stout prefix create ~/projects/api-service/.stout

# Install project dependencies
stout --prefix=~/projects/api-service/.stout bundle install

# Activate in shell
export PATH="$HOME/projects/api-service/.stout/bin:$PATH"
export STOUT_PREFIX="$HOME/projects/api-service/.stout"

Team-Shared Prefixes

# Create shared prefix
sudo stout prefix create /opt/team-data-science

# Set permissions
sudo chown -R :data-science /opt/team-data-science
sudo chmod -R g+w /opt/team-data-science

# Team members use
stout --prefix=/opt/team-data-science install pandas numpy scipy

Container Integration

Dockerfile:

FROM ubuntu:22.04

# Install stout
RUN curl -fsSL https://raw.githubusercontent.com/neul-labs/stout/main/install.sh | bash

# Create app-specific prefix
RUN stout prefix create /app/.stout

# Install dependencies
COPY Brewfile /app/
RUN stout --prefix=/app/.stout bundle install

# Add to PATH
ENV PATH="/app/.stout/bin:$PATH"
ENV STOUT_PREFIX="/app/.stout"

COPY . /app
WORKDIR /app
CMD ["./start.sh"]

Audit Logging

Track package operations for compliance and debugging.

Enable Audit Logging

~/.stout/config.toml:

[audit]
enabled = true
log_file = "/var/log/stout/audit.log"
log_format = "json"  # or "text"
include_user = true
include_timestamp = true

Log Format

JSON format:

{
  "timestamp": "2024-11-27T12:00:00Z",
  "user": "developer",
  "action": "install",
  "packages": ["jq@1.7.1"],
  "success": true,
  "duration_ms": 1234,
  "source": "bottle"
}

Centralized Logging

Send logs to your SIEM or log aggregator:

# Syslog
logger -t stout "$(cat /var/log/stout/audit.log)"

# Fluent Bit / Fluentd
# Configure to tail /var/log/stout/audit.log

# Datadog
# Use the Datadog agent with log collection

Compliance

SOC 2

stout supports SOC 2 compliance through:

  1. Access controls: Package installation requires appropriate permissions
  2. Audit trails: All operations logged with timestamps and users
  3. Change management: Version control via lock files
  4. Vulnerability management: Built-in stout audit command

HIPAA

For HIPAA-regulated environments:

  1. Air-gapped operation: No external network access required
  2. Encrypted storage: Compatible with encrypted filesystems
  3. Access logging: Track who installed what and when

FedRAMP

Federal deployments can use:

  1. Private hosting: No dependency on external services
  2. Custom signing: Own cryptographic keys
  3. FIPS compliance: Uses standard cryptographic algorithms

Performance at Scale

Caching Strategies

Shared Cache for Build Agents:

# NFS mount
mount -t nfs cache-server:/stout-cache /opt/stout-cache

# Configure stout
export STOUT_CACHE_DIR=/opt/stout-cache

Redis Cache for Metadata:

[cache]
backend = "redis"
redis_url = "redis://cache-server:6379/0"

Parallel Operations

# Increase parallel downloads
stout config set install.parallel_downloads 8

# Parallel builds (source installation)
stout install -s --jobs=16 large-package

Monitoring

Export metrics for monitoring:

# Prometheus metrics endpoint
stout metrics serve --port 9090

# Metrics available:
# - stout_packages_installed
# - stout_cache_hit_ratio
# - stout_download_duration_seconds
# - stout_install_duration_seconds

Quick Reference

Environment Variables

Variable Description Default
STOUT_PREFIX Installation prefix /opt/homebrew
STOUT_CACHE_DIR Cache directory ~/.stout/cache
STOUT_CONFIG Config file path ~/.stout/config.toml
STOUT_INDEX_URL Index URL override (from config)
STOUT_GITHUB_TOKEN GitHub auth token (none)
STOUT_SIGNING_KEY Signing key (CI) (none)
STOUT_LOG_LEVEL Log verbosity info
STOUT_NO_COLOR Disable colors false

CLI Flags for Enterprise

# Use specific prefix
stout --prefix=/custom/path install pkg

# Verbose output for debugging
stout -v install pkg
stout -vv install pkg  # More verbose

# Skip signature verification (development only)
stout update --insecure

# Dry run (show what would happen)
stout install --dry-run pkg

# Force operations
stout install --force pkg

For additional enterprise support, contact the maintainers or open a discussion on GitHub.