Skip to content

Expand FFI bindings with grammar, LoRA, embeddings, and more#1

Open
nbeerbower wants to merge 1 commit into
mainfrom
claude/expand-hemlock-ffi-pziin
Open

Expand FFI bindings with grammar, LoRA, embeddings, and more#1
nbeerbower wants to merge 1 commit into
mainfrom
claude/expand-hemlock-ffi-pziin

Conversation

@nbeerbower

Copy link
Copy Markdown
Contributor

New FFI bindings:

  • Grammar sampling (llama_sampler_init_grammar) for structured output
  • LoRA adapter support (load, apply, remove adapters)
  • State/session save and load functions
  • Additional samplers (XTC, top-n-sigma, infill, DRY)
  • FIM (Fill-in-Middle) tokens for code completion
  • Extended model metadata access
  • Chat template formatting
  • Embeddings extraction helpers
  • Control vectors support
  • Context settings (causal attention, warmup mode)
  • Additional vocab functions (sep, mask, score, attr)
  • NUMA and RPC support detection

New high-level API functions:

  • sampler_create_with_grammar() - Grammar-constrained sampling
  • lora_load/apply/clear() - LoRA adapter management
  • state_save/load() - Session persistence
  • chat_format() - Format messages with model template
  • extract_embeddings() - Get text embeddings as array
  • fim_prompt() - Create FIM prompts for code models
  • model_info_extended() - Extended model information
  • model_meta/model_meta_all() - Access GGUF metadata

New examples:

  • json_output.hml - Generate structured JSON
  • lora.hml - LoRA adapter usage
  • embeddings.hml - Text embeddings and similarity
  • fim.hml - Fill-in-middle code completion
  • session.hml - Save/restore conversation state

Constants added:

  • Token types and attributes
  • ROPE_SCALING_LONGROPE
  • GRAMMAR_JSON (built-in JSON grammar)

New FFI bindings:
- Grammar sampling (llama_sampler_init_grammar) for structured output
- LoRA adapter support (load, apply, remove adapters)
- State/session save and load functions
- Additional samplers (XTC, top-n-sigma, infill, DRY)
- FIM (Fill-in-Middle) tokens for code completion
- Extended model metadata access
- Chat template formatting
- Embeddings extraction helpers
- Control vectors support
- Context settings (causal attention, warmup mode)
- Additional vocab functions (sep, mask, score, attr)
- NUMA and RPC support detection

New high-level API functions:
- sampler_create_with_grammar() - Grammar-constrained sampling
- lora_load/apply/clear() - LoRA adapter management
- state_save/load() - Session persistence
- chat_format() - Format messages with model template
- extract_embeddings() - Get text embeddings as array
- fim_prompt() - Create FIM prompts for code models
- model_info_extended() - Extended model information
- model_meta/model_meta_all() - Access GGUF metadata

New examples:
- json_output.hml - Generate structured JSON
- lora.hml - LoRA adapter usage
- embeddings.hml - Text embeddings and similarity
- fim.hml - Fill-in-middle code completion
- session.hml - Save/restore conversation state

Constants added:
- Token types and attributes
- ROPE_SCALING_LONGROPE
- GRAMMAR_JSON (built-in JSON grammar)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants