|
| static bool | Initialize () |
| | Initialize the engine. Tries to find/load model, returns false if unavailable.
|
| |
| static bool | IsAvailable () |
| | Check if Gemma is available (model found or downloadable)
|
| |
| static GemmaActorResult | ExtractActors (const std::set< std::string > &sampleMessages) |
| |
| static GemmaDirectionResult | DetectDirections (const std::set< std::string > &sampleMessages) |
| |
| static std::string | GetModelPath () |
| | Get/set custom model path (default: ~/.logviewer/models/gemma-2b.gguf)
|
| |
| static void | SetModelPath (const std::string &path) |
| |
| static void | Shutdown () |
| | Shutdown the engine (frees resources)
|
| |
| static std::string | DownloadModel (const std::string &url, const std::string &filename) |
| |
| static bool | HasModel () |
| | Check if model is already downloaded.
|
| |
| static std::string | GetModelDownloadUrl () |
| | Get model download URL.
|
| |
Gemma 2B inference engine for local actor discovery. Manages model loading, downloading, and inference via llama.cpp.
| std::pair< std::string, std::string > ai::GemmaInferenceEngine::RunInference |
( |
const std::string & |
prompt, |
|
|
int |
maxTokens = 200 |
|
) |
| |
|
staticprivate |
Helper: Run inference on a prompt using the Gemma model Returns pair<output_text, error_message> - if error is empty, output is valid
Static helper: Run inference on a prompt using the Gemma model Returns pair<output_text, error_message> - if error is empty, output is valid
NOTE: This is a stub implementation. The actual inference loop requires:
- llama_tokenize(model, prompt, add_bos, special)
- llama_decode(ctx, batch) for context evaluation
- llama_sample_* functions for token selection
- llama_token_to_piece(model, token, buf, buf_size) for decoding The exact API depends on the llama.cpp version used.