Deployment and Operations

This document provides guidelines for deploying and operating GoVector in production environments. It covers deployment strategies, configuration options, monitoring, and best practices.

🚀 Deployment Options

Embedded Mode

Embedded mode is the simplest deployment option, where GoVector runs directly within your application process.

Use Cases

• Single-instance applications
• Edge devices
• Applications with low traffic
• Development and testing

Deployment Steps

1. Add as a dependency:

go get github.com/yourusername/govector

1. Initialize in your application:

import (
    "github.com/yourusername/govector/core"
)

// Initialize collection
collection, err := core.NewCollection(core.CollectionConfig{
    Name:       "my-collection",
    VectorLen:  768,
    Metric:     core.Cosine,
    IndexType:  core.HNSW,
    Quantize:   true, // Enable quantization for production
})
if err != nil {
    log.Fatalf("Failed to create collection: %v", err)
}

// Save collection on shutdown
defer func() {
    if err := collection.Save("/path/to/persistence"); err != nil {
        log.Printf("Failed to save collection: %v", err)
    }
}()

Microservice Mode

Microservice mode runs GoVector as a standalone server, accessible via HTTP API.

Use Cases

• Multi-application systems
• Language-agnostic environments
• High-traffic applications
• Centralized vector storage

Deployment Steps

1. Build the server:

go build -o govector-server ./cmd/server

1. Run the server:

./govector-server --host 0.0.0.0 --port 6333 --data ./data

1. Using systemd:

Create a systemd service file /etc/systemd/system/govector.service:

[Unit]
Description=GoVector Vector Database
After=network.target

[Service]
Type=simple
User=youruser
WorkingDirectory=/path/to/govector
ExecStart=/path/to/govector-server --host 0.0.0.0 --port 6333 --data ./data
Restart=on-failure

[Install]
WantedBy=multi-user.target

1. Start and enable the service:

sudo systemctl daemon-reload
sudo systemctl start govector
sudo systemctl enable govector

🔧 Configuration

Server Configuration

Flag	Description	Default
--host	Host to listen on	127.0.0.1
--port	Port to listen on	6333
--data	Data directory	./data
--log-level	Log level (debug, info, warn, error)	info

Environment Variables

Variable	Description	Default
GOVECTOR_HOST	Host to listen on	127.0.0.1
GOVECTOR_PORT	Port to listen on	6333
GOVECTOR_DATA_DIR	Data directory	./data
GOVECTOR_LOG_LEVEL	Log level	info

📊 Monitoring

Health Check

GoVector provides a health check endpoint:

curl http://localhost:6333/health

Metrics

GoVector exposes Prometheus-compatible metrics at http://localhost:6333/metrics:

• govector_collection_count - Number of collections
• govector_point_count{collection="name"} - Number of points per collection
• govector_search_requests_total - Total search requests
• govector_search_duration_seconds - Search latency
• govector_upsert_requests_total - Total upsert requests
• govector_upsert_duration_seconds - Upsert latency

Logging

GoVector logs to stdout/stderr by default. For production, consider redirecting logs to a centralized logging system:

./govector-server --host 0.0.0.0 --port 6333 --data ./data 2>&1 | tee -a /var/log/govector.log

🛡️ Security

Network Security

• Use HTTPS in production
• Limit network access to trusted clients
• Use a reverse proxy (e.g., Nginx) for additional security

Authentication

While GoVector doesn't include built-in authentication, you can add it using a reverse proxy:

server {
    listen 443 ssl;
    server_name govector.example.com;

    ssl_certificate /etc/ssl/certs/cert.pem;
    ssl_certificate_key /etc/ssl/private/key.pem;

    location / {
        proxy_pass http://localhost:6333;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;

        # Basic authentication
        auth_basic "Restricted";
        auth_basic_user_file /etc/nginx/.htpasswd;
    }
}

💾 Backup and Recovery

Backup

Regularly backup the data directory:

# Create backup
mkdir -p /backup/govector
cp -r /path/to/data/* /backup/govector/

# Compress backup
tar -czf /backup/govector-$(date +%Y%m%d).tar.gz /backup/govector/

Recovery

To recover from a backup:

1. Stop GoVector
2. Restore the backup:

# Restore from backup
tar -xzf /backup/govector-20231201.tar.gz -C /path/to/

# Set correct permissions
chown -R youruser:yourgroup /path/to/data

1. Start GoVector

📈 Performance Tuning

For High Traffic

• Use SSD storage for better I/O performance
• Enable SQ8 quantization to reduce memory usage
• Adjust HNSW parameters for your dataset size
• Use batch operations for bulk inserts

For Large Datasets

• Increase memory allocation
• Use multiple collections to distribute load
• Consider sharding across multiple instances

🚩 Common Issues

Out of Memory

• Cause: Large dataset without quantization
• Solution: Enable SQ8 quantization, increase memory allocation

Slow Performance

• Cause: Inappropriate HNSW parameters, slow disk
• Solution: Tune HNSW parameters, use SSD storage

Data Corruption

• Cause: Unexpected shutdown, disk error
• Solution: Regular backups, use journaling filesystem

🔗 Related Documentation

• Usage Modes - Different ways to use GoVector
• Collection Management - Collection creation and configuration
• HNSW Index - HNSW index implementation
• Storage Engine - Persistence and serialization