Troubleshooting¶

This document provides solutions to common issues you may encounter when using GoVector. It covers error messages, performance problems, and other frequently asked questions.

🚩 Common Error Messages¶

Dimension Mismatch¶

Error: dimension mismatch

Cause: The vector dimension doesn't match the collection's configured dimension.

Solution: - Ensure all vectors have the same dimension as the collection configuration - Check the collection's VectorLen parameter - Verify that your embedding model produces vectors of the correct dimension

Out of Memory¶

Error: out of memory

Cause: The collection is too large for the available memory.

Solution: - Enable SQ8 quantization to reduce memory usage by ~75% - Use a smaller M value for HNSW index - Consider using Flat index for small datasets - Increase the memory allocation for your application

Collection Not Found¶

Error: collection not found

Cause: The specified collection doesn't exist.

Solution: - Check the collection name for typos - Verify that the collection was created successfully - Ensure you're using the correct data directory

Point Not Found¶

Error: point not found

Cause: The specified point ID doesn't exist in the collection.

Solution: - Check the point ID for typos - Verify that the point was upserted successfully - Ensure you're using the correct collection

📊 Performance Issues¶

Slow Search¶

Symptom: Search operations are taking longer than expected.

Possible Causes: - Inappropriate HNSW parameters - Large result sets - Complex filters - Slow disk I/O

Solutions: - Increase EfSearch parameter for better recall (but slower search) - Decrease EfSearch parameter for faster search (but lower recall) - Use more selective filters to reduce result set size - Use SSD storage for better I/O performance - Enable SQ8 quantization to reduce memory usage

Slow Upsert¶

Symptom: Upsert operations are taking longer than expected.

Possible Causes: - Large batch sizes - High EfConstruction parameter - Slow disk I/O - Disabled quantization

Solutions: - Use appropriate batch sizes (100-1000 points per batch) - Adjust EfConstruction parameter based on dataset size - Use SSD storage for better I/O performance - For write-heavy workloads, consider disabling quantization

High Memory Usage¶

Symptom: GoVector is using more memory than expected.

Possible Causes: - Large dataset without quantization - High M value for HNSW index - Multiple collections loaded simultaneously

Solutions: - Enable SQ8 quantization to reduce memory usage - Use smaller M value for HNSW index - Load only necessary collections - Consider sharding across multiple instances

🔧 Configuration Issues¶

Incorrect Metric¶

Symptom: Search results don't match expected similarity.

Cause: The distance metric doesn't match your use case.

Solution: - Use Cosine for normalized vectors - Use Euclidean for unnormalized vectors - Use Dot for some specific use cases

Index Parameters¶

Symptom: Poor search performance or recall.

Cause: HNSW index parameters are not optimized for your dataset.

Solution: - For small datasets (10,000-100,000 points): M=12, EfConstruction=100 - For medium datasets (100,000-1,000,000 points): M=16, EfConstruction=200 - For large datasets (1,000,000+ points): M=24, EfConstruction=400

🛡️ Security Issues¶

Unauthorized Access¶

Symptom: Unauthorized users can access the API.

Cause: No authentication is configured.

Solution: - Use a reverse proxy with authentication - Restrict network access to trusted clients - Use HTTPS in production

Data Leakage¶

Symptom: Sensitive data is exposed.

Cause: Payload contains sensitive information.

Solution: - Don't store sensitive information in payloads - Use encryption for sensitive data - Implement proper access controls

💾 Storage Issues¶

Data Corruption¶

Symptom: Collection fails to load or returns incorrect data.

Cause: Unexpected shutdown, disk error, or corrupted files.

Solution: - Restore from backup - Use journaling filesystem - Implement proper error handling - Regularly verify data integrity

Disk Space Issues¶

Symptom: Ran out of disk space.

Cause: Large vectors, no quantization, or excessive payloads.

Solution: - Enable SQ8 quantization - Optimize payload size - Use smaller vectors when possible - Clean up old collections

📡 Network Issues¶

Connection Refused¶

Symptom: Cannot connect to the GoVector server.

Cause: - Server is not running - Incorrect host or port - Firewall blocking access

Solution: - Start the GoVector server - Verify host and port configuration - Check firewall settings

Timeout¶

Symptom: API requests timeout.

Cause: - Slow server performance - Network latency - Large result sets

Solution: - Optimize server performance - Use appropriate batch sizes - Reduce result set size - Check network connectivity

❓ Frequently Asked Questions¶

How to choose between HNSW and Flat index?¶

Answer: - Use HNSW for large datasets (10,000+ points) where search performance is important - Use Flat for small datasets (< 10,000 points) where accuracy is critical

How to optimize memory usage?¶

Answer: - Enable SQ8 quantization - Use smaller M value for HNSW index - Load only necessary collections - Consider using Flat index for small datasets

How to back up GoVector data?¶

Answer: - Regularly copy the data directory - Use automated backup tools - Test restoration process periodically

How to scale GoVector?¶

Answer: - Use multiple collections to distribute load - Consider sharding across multiple instances - Use load balancing for microservice mode - Optimize HNSW parameters for your dataset

How to integrate with other languages?¶

Answer: - Use microservice mode with HTTP API - Use existing Qdrant clients - Implement custom clients for specific languages

Deployment - Deployment and operations
Collection Management - Collection creation and configuration
HNSW Index - HNSW index implementation
Storage Engine - Persistence and serialization