- Introduction
- Key Features
- System Architecture
- Installation and Setup
- Configuration
- API Reference
- Usage Examples
- Performance Optimization
- Security Considerations
- Monitoring and Logging
- Testing Strategy
- Deployment
- Contributing
- Versioning
- License
- Acknowledgments
- Support and Contact
tod-p2m (Torrent-On-Demand Streaming Platform) is a high-performance, scalable application designed for streaming media content directly from torrent files. Built with Go, it leverages the power of BitTorrent protocol to provide seamless, on-demand access to a wide range of media formats including video, audio, images, and documents.
The primary goal of tod-p2m is to offer a robust, efficient, and user-friendly solution for streaming torrent-based content, eliminating the need for complete downloads before playback. This platform is ideal for applications requiring quick access to large media files, such as video-on-demand services, digital libraries, or content distribution networks.
- Software developers building media streaming applications
- System administrators managing content delivery networks
- Researchers working on peer-to-peer technologies
- Media companies looking for efficient content distribution solutions
- Magnet Link Support: Seamless integration with magnet links for instant content access.
- Multi-Protocol Streaming:
- HTTP Live Streaming (HLS) for adaptive bitrate streaming
- HTTP byte-range requests for progressive downloading
- Format Versatility: Support for various media formats including video (MP4, WebM), audio (MP3, OGG), images (JPEG, PNG), and documents (PDF, TXT).
- Dynamic Rate Limiting: Configurable bandwidth management for both download and upload streams.
- Intelligent Caching: Sophisticated caching mechanisms to optimize resource usage and enhance performance.
- Cross-Origin Resource Sharing (CORS): Built-in CORS support for seamless integration with web applications.
- Concurrent Connection Handling: Efficient management of multiple simultaneous connections leveraging Go's concurrency model.
- Real-time Analytics: Detailed metrics and analytics for monitoring system performance and user engagement.
tod-p2m is built on a modular, microservices-oriented architecture, ensuring scalability, maintainability, and ease of deployment.
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ API Gateway │ │ Torrent Engine │ │ Media Streamer │
│ │ │ │ │ │
│ ┌───────────┐ │ │ ┌───────────┐ │ │ ┌───────────┐ │
│ │ Rate │ │ │ │ Torrent │ │ │ │ HLS │ │
│ │ Limiter │◄─┼───┼─▶│ Manager │◄─┼───┼─▶│ Generator │ │
│ └───────────┘ │ │ └───────────┘ │ │ └───────────┘ │
│ │ │ │ │ │
│ ┌───────────┐ │ │ ┌───────────┐ │ │ ┌───────────┐ │
│ │ Auth │ │ │ │ Piece │ │ │ │ MIME │ │
│ │ Middleware│ │ │ │ Selector │ │ │ │ Handler │ │
│ └───────────┘ │ │ └───────────┘ │ │ └───────────┘ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
▲ ▲ ▲
│ │ │
└─────────────────────┼─────────────────────┘
│
┌───────────────────────┐
│ Storage&DB │
└───────────────────────┘
- API Gateway: Handles incoming requests, authentication, and rate limiting.
- Torrent Engine: Manages torrent downloads, piece selection, and peer communication.
- Media Streamer: Responsible for adaptive streaming and format-specific handling.
- Database: Stores metadata and caching data.
- Backend: Go 1.18+
- Frameworks:
- HTTP Router: Go-Chi
- Torrent Client: anacrolix/torrent
- Logging: zerolog
- Database: Redis for caching, PostgreSQL for persistent storage
- Containerization: Docker
- Orchestration: Kubernetes (optional for large-scale deployments)
- Go 1.18 or higher
- Docker and Docker Compose (for containerized deployment)
- Git
-
Clone the repository:
git clone https://github.com/osvalois/tod-p2m.git cd tod-p2m
-
Install dependencies:
go mod download
-
Build the application:
go build -o tod-p2m ./cmd/server
-
Run the application:
./tod-p2m
-
Build the Docker image:
docker build -t tod-p2m:latest .
-
Run the container:
docker run -p 8080:8080 -v /path/to/config:/app/config tod-p2m:latest
tod-p2m uses a YAML configuration file and environment variables for flexible setup.
server:
port: 8080
host: "0.0.0.0"
logging:
level: "info"
format: "json"
torrent:
max_connections: 100
download_rate_limit: 0 # 0 means unlimited
upload_rate_limit: 0
cache_size: 100 # Number of torrents to keep in cache
cleanup_interval: "10m"
streaming:
hls_segment_duration: 10
buffer_size: 1048576 # 1MB
security:
enable_cors: true
allowed_origins: ["*"]
database:
type: "postgres"
dsn: "postgres://user:password@localhost/tod_p2m?sslmode=disable"
redis:
address: "localhost:6379"
password: ""
db: 0
Environment variables override config file settings. Use the prefix TOD_P2M_
for all variables.
Example:
export TOD_P2M_SERVER_PORT=9090
export TOD_P2M_LOGGING_LEVEL=debug
Detailed API documentation is available in the API Reference document. Here's a summary of the main endpoints:
GET /torrent/{infoHash}
: Retrieve torrent metadataGET /stream/{infoHash}/{fileID}
: Stream a file via HTTPGET /hls/{infoHash}/{fileID}/playlist.m3u8
: Get HLS playlistGET /hls/{infoHash}/{fileID}/{segmentID}.ts
: Get HLS segmentGET /document/{infoHash}/{fileID}
: Stream a documentGET /image/{infoHash}/{fileID}
: Stream an image
const videoPlayer = document.getElementById('video-player');
const infoHash = '1234567890abcdef1234567890abcdef12345678';
const fileID = 0;
videoPlayer.src = `http://localhost:8080/hls/${infoHash}/${fileID}/playlist.m3u8`;
videoPlayer.play();
curl http://localhost:8080/torrent/1234567890abcdef1234567890abcdef12345678
Response:
{
"infoHash": "1234567890abcdef1234567890abcdef12345678",
"name": "Big Buck Bunny",
"files": [
{
"id": 0,
"name": "big_buck_bunny.mp4",
"size": 276445467,
"progress": 0.15
}
]
}
tod-p2m implements several strategies to ensure high performance and efficient resource utilization:
- In-memory caching of frequently accessed torrent metadata
- Disk caching of popular content pieces to reduce network overhead
- Reuse of BitTorrent peer connections to minimize handshake overhead
- Database connection pooling for efficient resource management
- Non-blocking I/O operations leveraging Go's goroutines
- Parallel processing of multiple torrent pieces
- Round-robin DNS for distributing incoming requests across multiple instances
- Consistent hashing for efficient content distribution in clustered deployments
All user inputs, including infoHashes and file IDs, are strictly validated to prevent injection attacks.
IP-based rate limiting is implemented to prevent abuse and ensure fair usage.
HTTPS is strongly recommended for production deployments. Refer to HTTPS Setup Guide for implementation details.
Implement appropriate authentication and authorization mechanisms when deploying in production environments.
tod-p2m uses structured logging with zerolog. Log levels are configurable, and logs can be easily integrated with log aggregation systems like ELK stack or Splunk.
Key metrics are exposed via a /metrics
endpoint in Prometheus format, including:
- Active connections
- Torrents in cache
- Bandwidth usage
- Request latencies
A /health
endpoint is provided for monitoring the application's health status.
Run unit tests with:
go test ./...
Run integration tests with:
go test -tags=integration ./...
Refer to the Load Testing Guide for instructions on performing load tests using tools like Apache JMeter or Gatling.
Refer to the Docker Deployment Guide for detailed instructions on deploying tod-p2m using Docker.
We welcome contributions to tod-p2m! Please refer to our Contributing Guidelines for details on our code of conduct and the process for submitting pull requests.
We use SemVer for versioning. For the versions available, see the tags on this repository.
This project is licensed under the MIT License - see the LICENSE file for details.
- The anacrolix/torrent library for providing the core BitTorrent functionality.
- The Go community for their excellent libraries and tools.
For support, please open an issue in the GitHub repository or contact the maintainers at osvaloismtz@gmail.com.
tod-p2m is continually evolving to meet the demands of modern content delivery. We encourage community feedback and contributions to help make this project even better.