server

package
v0.0.0-...-acc130e Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 30, 2026 License: Apache-2.0 Imports: 19 Imported by: 0

Documentation

Overview

Package server provides the HTTP server for the web UI and REST API.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ApplyConfigToServer 

func ApplyConfigToServer(serverCfg *Config) error

ApplyConfigToServer applies config file settings to server config. CLI flags take precedence (non-zero values are not overwritten).

func ConfigPath 

func ConfigPath() string

ConfigPath returns the path to the config file. Checks in order: hfdownloader.json, hfdownloader.yaml, hfdownloader.yml

func SaveConfigFile 

func SaveConfigFile(cfg *ConfigFile) error

SaveConfigFile saves configuration to the config file.

Types

type CacheStats 

type CacheStats struct {
	TotalModels    int    `json:"totalModels"`
	TotalDatasets  int    `json:"totalDatasets"`
	TotalSize      int64  `json:"totalSize"`
	TotalSizeHuman string `json:"totalSizeHuman"`
	TotalFiles     int    `json:"totalFiles"`
}

CacheStats contains aggregate statistics about the cache.

type CachedFileInfo 

type CachedFileInfo struct {
	Name      string `json:"name"`
	Size      int64  `json:"size"`
	SizeHuman string `json:"sizeHuman"`
	IsLFS     bool   `json:"isLfs"`
}

CachedFileInfo represents a file in the cache.

type CachedRepoInfo 

type CachedRepoInfo struct {
	Repo           string           `json:"repo"`
	Owner          string           `json:"owner"`
	Name           string           `json:"name"`
	Type           string           `json:"type"` // "model" or "dataset"
	Path           string           `json:"path"`
	FriendlyPath   string           `json:"friendlyPath,omitempty"`
	Size           int64            `json:"size"`
	SizeHuman      string           `json:"sizeHuman"`
	FileCount      int              `json:"fileCount"`
	Branch         string           `json:"branch,omitempty"`
	Commit         string           `json:"commit,omitempty"`
	Downloaded     string           `json:"downloaded,omitempty"`
	DownloadStatus string           `json:"downloadStatus,omitempty"` // "complete", "filtered", "unknown"
	Snapshots      []string         `json:"snapshots,omitempty"`
	Files          []CachedFileInfo `json:"files,omitempty"`
	Manifest       *ManifestInfo    `json:"manifest,omitempty"`
}

CachedRepoInfo represents a cached repository for the API response.

type Config 

type Config struct {
	Addr               string
	Port               int
	Token              string // HuggingFace token
	ModelsDir          string // Output directory for models (not configurable via API)
	DatasetsDir        string // Output directory for datasets (not configurable via API)
	CacheDir           string // HuggingFace cache directory for v3 mode
	Concurrency        int
	MaxActive          int
	MultipartThreshold string   // Minimum size for multipart download
	Verify             string   // Verification mode: none, size, sha256
	Retries            int      // Number of retry attempts
	AllowedOrigins     []string // CORS origins
	Endpoint           string   // Custom HuggingFace endpoint (e.g., for mirrors)

	// Authentication
	AuthUser string // Basic auth username (empty = no auth)
	AuthPass string // Basic auth password

	// Proxy configuration
	Proxy *hfdownloader.ProxyConfig
}

Config holds server configuration.

func DefaultConfig 

func DefaultConfig() Config

DefaultConfig returns sensible defaults.

type ConfigFile 

type ConfigFile struct {
	CacheDir           string       `json:"cache-dir,omitempty" yaml:"cache-dir,omitempty"`
	Token              string       `json:"token,omitempty" yaml:"token,omitempty"`
	Connections        int          `json:"connections,omitempty" yaml:"connections,omitempty"`
	MaxActive          int          `json:"max-active,omitempty" yaml:"max-active,omitempty"`
	MultipartThreshold string       `json:"multipart-threshold,omitempty" yaml:"multipart-threshold,omitempty"`
	Verify             string       `json:"verify,omitempty" yaml:"verify,omitempty"`
	Retries            int          `json:"retries,omitempty" yaml:"retries,omitempty"`
	Endpoint           string       `json:"endpoint,omitempty" yaml:"endpoint,omitempty"`
	BackoffInitial     string       `json:"backoff-initial,omitempty" yaml:"backoff-initial,omitempty"`
	BackoffMax         string       `json:"backoff-max,omitempty" yaml:"backoff-max,omitempty"`
	Proxy              *ProxyConfig `json:"proxy,omitempty" yaml:"proxy,omitempty"`
}

ConfigFile represents the persistent configuration file format. This matches the CLI config file format for consistency.

func LoadConfigFile 

func LoadConfigFile() (*ConfigFile, error)

LoadConfigFile loads configuration from the config file. Returns empty config if file doesn't exist (not an error).

type DownloadRequest 

type DownloadRequest struct {
	Repo               string   `json:"repo"`
	Revision           string   `json:"revision,omitempty"`
	Dataset            bool     `json:"dataset,omitempty"`
	Filters            []string `json:"filters,omitempty"`
	Excludes           []string `json:"excludes,omitempty"`
	AppendFilterSubdir bool     `json:"appendFilterSubdir,omitempty"`
	DryRun             bool     `json:"dryRun,omitempty"`
}

DownloadRequest is the request body for starting a download. Note: Output path is NOT configurable via API for security reasons. The server uses its configured OutputDir (Models/ for models, Datasets/ for datasets).

type ErrorResponse 

type ErrorResponse struct {
	Error   string `json:"error"`
	Details string `json:"details,omitempty"`
}

ErrorResponse represents an API error.

type Job 

type Job struct {
	ID        string            `json:"id"`
	Repo      string            `json:"repo"`
	Revision  string            `json:"revision"`
	IsDataset bool              `json:"isDataset,omitempty"`
	Filters   []string          `json:"filters,omitempty"`
	Excludes  []string          `json:"excludes,omitempty"`
	OutputDir string            `json:"outputDir"`
	Status    JobStatus         `json:"status"`
	Progress  JobProgress       `json:"progress"`
	Error     string            `json:"error,omitempty"`
	CreatedAt time.Time         `json:"createdAt"`
	StartedAt *time.Time        `json:"startedAt,omitempty"`
	EndedAt   *time.Time        `json:"endedAt,omitempty"`
	Files     []JobFileProgress `json:"files,omitempty"`
	// contains filtered or unexported fields
}

Job represents a download job.

type JobFileProgress 

type JobFileProgress struct {
	Path       string `json:"path"`
	TotalBytes int64  `json:"totalBytes"`
	Downloaded int64  `json:"downloaded"`
	Status     string `json:"status"` // pending, active, complete, skipped, error
}

JobFileProgress holds per-file progress.

type JobManager 

type JobManager struct {
	// contains filtered or unexported fields
}

JobManager manages download jobs.

func NewJobManager 

func NewJobManager(cfg Config, wsHub *WSHub) *JobManager

NewJobManager creates a new job manager.

func (*JobManager) CancelJob 

func (m *JobManager) CancelJob(id string) bool

CancelJob cancels a running or queued job.

func (*JobManager) CreateJob 

func (m *JobManager) CreateJob(req DownloadRequest) (*Job, bool, error)

CreateJob creates a new download job. Returns existing job if same repo+revision+dataset is already in progress.

func (*JobManager) DeleteJob 

func (m *JobManager) DeleteJob(id string) bool

DeleteJob removes a job from the list.

func (*JobManager) GetJob 

func (m *JobManager) GetJob(id string) (*Job, bool)

GetJob retrieves a job by ID.

func (*JobManager) ListJobs 

func (m *JobManager) ListJobs() []*Job

ListJobs returns all jobs.

func (*JobManager) PauseJob 

func (m *JobManager) PauseJob(id string) bool

PauseJob pauses a running job.

func (*JobManager) ResumeJob 

func (m *JobManager) ResumeJob(id string) bool

ResumeJob resumes a paused job.

func (*JobManager) Subscribe 

func (m *JobManager) Subscribe() chan *Job

Subscribe adds a listener for job updates.

func (*JobManager) Unsubscribe 

func (m *JobManager) Unsubscribe(ch chan *Job)

Unsubscribe removes a listener.

type JobProgress 

type JobProgress struct {
	TotalFiles      int   `json:"totalFiles"`
	CompletedFiles  int   `json:"completedFiles"`
	TotalBytes      int64 `json:"totalBytes"`
	DownloadedBytes int64 `json:"downloadedBytes"`
	BytesPerSecond  int64 `json:"bytesPerSecond"`
}

JobProgress holds aggregate progress info.

type JobStatus 

type JobStatus string

JobStatus represents the state of a download job. 

const (
	JobStatusQueued    JobStatus = "queued"
	JobStatusRunning   JobStatus = "running"
	JobStatusPaused    JobStatus = "paused"
	JobStatusCompleted JobStatus = "completed"
	JobStatusFailed    JobStatus = "failed"
	JobStatusCancelled JobStatus = "cancelled"
)

type ManifestInfo 

type ManifestInfo struct {
	Branch     string `json:"branch"`
	Commit     string `json:"commit"`
	Downloaded string `json:"downloaded"`
	Command    string `json:"command,omitempty"`
	TotalSize  int64  `json:"totalSize"`
	TotalFiles int    `json:"totalFiles"`
	IsFiltered bool   `json:"isFiltered"`        // True if download used filters
	Filters    string `json:"filters,omitempty"` // The filter string if used
}

ManifestInfo contains manifest data if available.

type MirrorDiffEntry 

type MirrorDiffEntry struct {
	Repo       string `json:"repo"`
	Type       string `json:"type"`
	Status     string `json:"status"` // "missing", "outdated", "extra"
	LocalSize  int64  `json:"localSize,omitempty"`
	RemoteSize int64  `json:"remoteSize,omitempty"`
	SizeHuman  string `json:"sizeHuman,omitempty"`
}

MirrorDiffEntry represents a difference between source and target.

type MirrorDiffRequest 

type MirrorDiffRequest struct {
	Target     string `json:"target"`     // Target name or path
	RepoFilter string `json:"repoFilter"` // Optional filter by repo name
}

MirrorDiffRequest is the request body for diff operation.

type MirrorSyncRequest 

type MirrorSyncRequest struct {
	Target      string `json:"target"`      // Target name or path
	RepoFilter  string `json:"repoFilter"`  // Optional filter by repo name
	DryRun      bool   `json:"dryRun"`      // Show what would be done without doing it
	Verify      bool   `json:"verify"`      // Verify integrity after copy
	DeleteExtra bool   `json:"deleteExtra"` // Delete repos in destination not in source
	Force       bool   `json:"force"`       // Re-copy incomplete/outdated repos
}

MirrorSyncRequest is the request body for push/pull operations.

type MirrorSyncResult 

type MirrorSyncResult struct {
	Success          bool     `json:"success"`
	DryRun           bool     `json:"dryRun"`
	Copied           int      `json:"copied"`
	CopiedSize       int64    `json:"copiedSize"`
	CopiedSizeHuman  string   `json:"copiedSizeHuman"`
	Deleted          int      `json:"deleted"`
	DeletedSize      int64    `json:"deletedSize"`
	DeletedSizeHuman string   `json:"deletedSizeHuman"`
	Repos            []string `json:"repos,omitempty"`
	Errors           []string `json:"errors,omitempty"`
	Message          string   `json:"message"`
}

MirrorSyncResult represents the result of a sync operation.

type MirrorTargetRequest 

type MirrorTargetRequest struct {
	Name        string `json:"name"`
	Path        string `json:"path"`
	Description string `json:"description,omitempty"`
}

MirrorTargetRequest is the request body for adding a mirror target.

type PlanFile 

type PlanFile struct {
	Path string `json:"path"`
	Size int64  `json:"size"`
	LFS  bool   `json:"lfs"`
}

PlanFile represents a file in the plan.

type PlanResponse 

type PlanResponse struct {
	Repo       string     `json:"repo"`
	Revision   string     `json:"revision"`
	Files      []PlanFile `json:"files"`
	TotalSize  int64      `json:"totalSize"`
	TotalFiles int        `json:"totalFiles"`
}

PlanResponse is the response for a dry-run/plan request.

type ProxyConfig 

type ProxyConfig struct {
	URL                string `json:"url,omitempty" yaml:"url,omitempty"`
	Username           string `json:"username,omitempty" yaml:"username,omitempty"`
	Password           string `json:"password,omitempty" yaml:"password,omitempty"`
	NoProxy            string `json:"no_proxy,omitempty" yaml:"no_proxy,omitempty"`
	NoEnvProxy         bool   `json:"no_env_proxy,omitempty" yaml:"no_env_proxy,omitempty"`
	InsecureSkipVerify bool   `json:"insecure_skip_verify,omitempty" yaml:"insecure_skip_verify,omitempty"`
}

ProxyConfig holds proxy settings for the config file.

type ProxySettingsResponse 

type ProxySettingsResponse struct {
	URL                string `json:"url,omitempty"`
	Username           string `json:"username,omitempty"`
	NoProxy            string `json:"noProxy,omitempty"`
	NoEnvProxy         bool   `json:"noEnvProxy,omitempty"`
	InsecureSkipVerify bool   `json:"insecureSkipVerify,omitempty"`
}

ProxySettingsResponse represents proxy configuration in API responses.

type RebuildResponse 

type RebuildResponse struct {
	Success         bool     `json:"success"`
	ReposScanned    int      `json:"reposScanned"`
	SymlinksCreated int      `json:"symlinksCreated"`
	SymlinksUpdated int      `json:"symlinksUpdated"`
	OrphansRemoved  int      `json:"orphansRemoved,omitempty"`
	Errors          []string `json:"errors,omitempty"`
	Message         string   `json:"message,omitempty"`
}

RebuildResponse represents the result of a cache rebuild operation.

type Server 

type Server struct {
	// contains filtered or unexported fields
}

Server is the HTTP server for hfdownloader.

func New 

func New(cfg Config) *Server

New creates a new server with the given configuration.

func (*Server) ListenAndServe 

func (s *Server) ListenAndServe(ctx context.Context) error

ListenAndServe starts the HTTP server.

type SettingsResponse 

type SettingsResponse struct {
	Token              string `json:"token,omitempty"`
	CacheDir           string `json:"cacheDir"`
	Concurrency        int    `json:"connections"`
	MaxActive          int    `json:"maxActive"`
	MultipartThreshold string `json:"multipartThreshold"`
	Verify             string `json:"verify"`
	Retries            int    `json:"retries"`
	Endpoint           string `json:"endpoint,omitempty"`
	// Proxy settings
	Proxy *ProxySettingsResponse `json:"proxy,omitempty"`
	// Config file paths
	ConfigFile  string `json:"configFile,omitempty"`
	TargetsFile string `json:"targetsFile,omitempty"`
}

SettingsResponse represents current settings.

type SuccessResponse 

type SuccessResponse struct {
	Success bool   `json:"success"`
	Message string `json:"message,omitempty"`
}

SuccessResponse represents a simple success message.

type WSClient 

type WSClient struct {
	// contains filtered or unexported fields
}

WSClient represents a connected WebSocket client.

type WSHub 

type WSHub struct {
	// contains filtered or unexported fields
}

WSHub manages WebSocket clients and broadcasts.

func NewWSHub 

func NewWSHub() *WSHub

NewWSHub creates a new WebSocket hub.

func (*WSHub) Broadcast 

func (h *WSHub) Broadcast(msgType string, data any)

Broadcast sends a message to all connected clients.

func (*WSHub) BroadcastEvent 

func (h *WSHub) BroadcastEvent(event any)

BroadcastEvent sends a progress event to all clients.

func (*WSHub) BroadcastJob 

func (h *WSHub) BroadcastJob(job *Job)

BroadcastJob sends a job update to all clients.

func (*WSHub) ClientCount 

func (h *WSHub) ClientCount() int

ClientCount returns the number of connected clients.

func (*WSHub) Run 

func (h *WSHub) Run()

Run starts the hub's main loop.

type WSMessage 

type WSMessage struct {
	Type string `json:"type"`
	Data any    `json:"data"`
}

WSMessage represents a message sent over WebSocket.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.