edge_text_to_sql() turns a natural language question plus a CREATE TABLE
schema into SQL. Pass a DBI connection to con and it executes the query
and returns the rows, otherwise it returns the statement.
edge_narrate() writes a short English summary of a record. Accepts a
named list, a single row, or a multi-row data frame, and returns one
narrative per record.
edge_verify_narrative() checks a generated narrative against the values
it came from. A deterministic check, which needs no model, confirms that
every numeric source value appears in the text. With terms (for example
the study's drug names and MedDRA terms) it also flags terms the narrative
adds that are not in the source, likely hallucinations, and source terms it
leaves out. A model-based check re-extracts the named fields and compares
them to the source, with a tolerance for numeric fields. Both run by
default; method = "literal" runs the deterministic check alone.
edge_extract() and edge_extract_batch() gain draft = TRUE, which lets
the model answer in free text before the grammar-constrained pass. Forcing
a very small model straight into strict JSON can cost accuracy; drafting
first usually recovers it, at two to four times the generation time.
All three need a capable model. Small general purpose models produce valid SQL that answers a different question, and narratives that restate the input rather than summarise it. The reference pages say so.
edge_load_model() gains prompt_cache. With prompt_cache = TRUE the
model keeps its KV cache between calls and reuses the part of a new prompt
that matches the start of the previous one, so only the rest is processed.
It helps most when prompts share a long fixed beginning such as a system
prompt; in one test it cut each call by about 30%. Answers still depend
only on their own prompt. Off by default.
Grammar-constrained generation is about twice as fast on SIMD builds
(edge_grammar_completion() and everything built on it, such as
edge_extract() and edge_classify()). The grammar used to be checked
against every token in the vocabulary at every step; it now checks the
sampled token and falls back to the full check only when that token is
rejected, as llama.cpp does.
Missing standard library includes. Twenty three C library headers were used but not included, in files that only compiled because another header happened to pull them in. Two headers shared between C and C++ used the C++ spelling.
SIMD builds no longer crash on Windows. Installing with
EDGEMODELR_SIMD set to AVX, AVX2, AVX512 or NATIVE built fine but
segfaulted in edge_load_model(): 64-bit Windows GCC cannot keep the
stack 32-byte aligned for AVX, so aligned vector moves faulted. SIMD
builds on Windows now assemble with -Wa,-muse-unaligned-vector-move.
The default generic build is unchanged.
temperature = 0 now means greedy decoding, so the same prompt always
gives the same text. It used to skip temperature scaling but still sample
at random.
edge_json_grammar() allows at most one newline and indentation between
fields. With unlimited whitespace a model could fill n_predict with
blank lines and return incomplete JSON, which greedy decoding did every
time.
docker/ with a base image, a variant with TinyLlama baked in, and
a compose file. The base image declares the model cache as a volume so a
host cache can be mounted for offline use.Each completion now starts from an empty context. The KV cache was
never cleared between calls, so every prompt was appended after the
previous prompt and answer. Answers depended on earlier calls, and after
a few calls the context filled up and every later call failed with
"Failed to process prompt". This affected edge_completion(),
edge_stream_completion(), edge_grammar_completion() and the functions
built on them.
Generation can be interrupted. Esc/Ctrl-C now stops a long
generation, and timeout_seconds in edge_completion() works: it used
to have no effect until the call finished. A timeout raises a "timed
out" error. The sampler is freed on every exit path.
edge_index_documents() no longer hangs. Splitting any document
longer than chunk_size looped forever. chunk_overlap must now be
smaller than chunk_size.
edge_download_model() checks checksums for model names. With a name
from edge_list_models(), expected_sha256, verify_checksum and
trust_first_use were silently ignored.
trust_first_use = TRUE no longer fails after a download by URL.
Storing the hash raised an error once the file had been downloaded.
edge_install_cuda() no longer times out. The ~200 MB download used
R's default 60 second timeout.
edge_clean_cache() keeps the CUDA backend and checksum store. Both
were deleted once they were older than max_age_days. The size limit
now removes the oldest files until the cache is under the limit; before,
some caches could never get under it.
Prompts longer than the batch size crashed the session. The prompt was
passed to the model in a single call, which llama.cpp only accepts up to
n_batch tokens (512 with the default n_ctx = 2048), so any prompt
between that and the context size hit an internal assertion, for example
a long edge_ask() context. The prompt is now fed in n_batch-sized
chunks in completion, streaming and grammar-constrained generation.
Internal llama.cpp assertions no longer kill the R session. No ggml
abort callback was ever installed, so any GGML_ASSERT/GGML_ABORT hit
on R's thread ended in raise(SIGABRT). These now surface as an
ordinary R error ("llama.cpp internal error: ...") and the session keeps
working. Assertions on ggml worker threads still abort, because R cannot
be signalled from another thread.
Loading a file that is not GGUF crashed the R session. The bundled
loader is not hardened against arbitrary input, and on clang 23 a plain
text file made it read unmapped memory rather than return null, taking
the session down. edge_load_model() now checks for the GGUF magic
header before calling the loader and raises a normal R error instead.
The crash was reachable in every earlier release; it only surfaced now
because 0.4.1 failed to install on clang 23, so its tests never ran.
-Wkeyword-macro in ggml/ggml-common.h. C23 made static_assert
a keyword, and the header defined a macro over it. Valid C, but clang 23
warns and R CMD check counts that as a significant warning. The macro is
now skipped when the compiler is C23 or later.
Missing <cstdlib> in four more translation units: llama-mmap.cpp
(posix_memalign, free), llama-model-loader.cpp (getenv, atoi),
llama-vocab.cpp (strtol) and llama-batch.cpp (getenv, atoi,
malloc, free) used C library functions without including the header
that declares them. Recent libc++ releases no longer supply these
through transitive includes, so clang 23 and fedora-clang rejected them.
llama-batch.cpp was not in the compiler report, because the build
stopped before reaching it; it was found by auditing every file in
src/ for this defect rather than by patching the reported lines.
38 further missing standard includes across 26 files: the first audit
covered only the C headers (<cstdlib>, <cstring> and so on), but the
libc++ change behind the clang 23 failures removes transitive includes of
C++ headers too. A second audit covering the C++ standard library found
std::min, std::copy, std::advance, std::distance, std::unique_ptr,
std::is_same, std::out_of_range, std::array, std::pair, std::move
and isspace/toupper used in files that never included <algorithm>,
<iterator>, <memory>, <type_traits>, <stdexcept>, <array>,
<utility> or <cctype>. These build under libstdc++, which supplies the
declarations transitively, and would have failed under libc++.
False -Warray-bounds in llama-grammar.cpp: with optimisation and
-Wall now applied to the vendored sources, GCC 12 and later reported a
false positive against a zero-length array when
llama_grammar_advance_stack() copied a vector it had already proven
empty. R CMD check treats that warning as significant. The branch now
default-constructs the empty stack instead of copying it, which is
equivalent and does not trip the warning.
CFLAGS and CXXFLAGS now reach the whole package: the pattern
rules for ggml/ and llama/ passed only package-local flags to the
compiler, so flags set by the user or the site Makevars were applied
to 6 of 161 source files and silently dropped for the other 155.
Every rule in src/Makevars and src/Makevars.win now appends its
engine-specific additions to $(ALL_CFLAGS) or $(ALL_CXXFLAGS)
instead of replacing them. The variables were renamed to
GGML_EXTRA_* and GENERIC_EXTRA_* to make that contract explicit.
Installation failure on macOS x86_64 (CRAN r-release-macos-x86_64):
path_str() in ggml/ggml-backend-reg.cpp chose between two
conversion branches using #if defined(__cpp_lib_char8_t). The libc++
shipped in MacOSX11.sdk defines that macro under -std=gnu++20 but
its std::filesystem::path::u8string() still returns std::string,
so the branch guarded for std::u8string failed to compile. The
function now deduces the return type with auto and converts
byte-wise, which is correct whether the element type is char or
char8_t.
C++ standard pinned to C++17 (CXX_STD = CXX17 in src/Makevars and
src/Makevars.win): the bundled llama.cpp and GGML sources target C++17.
Without a pin, each platform applied its own default, so the same source
compiled as C++17 under R 4.5 and as C++20 under R 4.6 on macOS. Every
CRAN flavour now builds the dialect the vendored engine is written for.
Installation failure on clang 23: three translation units used
symbols they never included a header for, relying on transitive
includes that newer libc++ releases no longer provide.
ggml/gguf.cpp calls errno without <cerrno>, and
llama/llama-graph.h and llama/llama-context.cpp call getenv()
and atoi() without <cstdlib>. Each file now includes what it uses.
ggml/ggml.c and
ggml/ggml-opt.cpp. Both now include the same #ifdef USING_R
macro block that neutralizes printf, fprintf, fputs, fflush,
stderr, and stdout. These calls were diagnostic-only and were
already silent at runtime via the installed log callback; now the
symbols never reach the compiled object files either.Grammar-constrained generation (edge_grammar_completion()): Force model
output to conform to a GBNF grammar specification. Ensures valid, parseable
structured output (JSON, enums, numbers, etc.) using llama.cpp's native
grammar sampler.
JSON schema helper (edge_json_grammar()): Convert a simple R list
schema into a GBNF grammar string. Supports string, number, integer,
boolean fields and enum (character vector) constraints.
Structured data extraction (edge_extract()): High-level function that
combines prompt construction with grammar-constrained generation to extract
structured data from text. Returns a parsed R list (requires jsonlite).
Text classification (edge_classify()): Classify text into predefined
categories using grammar constraints. Supports single text and batch
(vectorized) classification. Output is guaranteed to be one of the
specified categories.
Text embeddings (edge_embeddings()): Extract dense vector embeddings
from any loaded model. Returns a numeric matrix (n_texts x n_embd) suitable
for clustering, semantic search, similarity computation, and RAG pipelines.
Supports optional L2 normalization.
Cosine similarity (edge_similarity(), edge_similarity_matrix()):
Compute pairwise cosine similarity between embedding vectors. Matrix version
efficiently computes all-pairs similarity using normalized matrix multiply.
Embedding dimension query (edge_model_n_embd()): Query the embedding
dimension of a loaded model.
Batch processing (edge_map()): Apply a prompt template over a vector
of texts with progress reporting. Supports both string templates with
{text} placeholder and custom prompt functions. Optional grammar
constraint for structured batch output.
Batch extraction (edge_extract_batch()): Extract structured data from
multiple texts, returning a data frame with one row per input.
RAG document indexing (edge_index_documents()): Build a semantic
embedding index from a directory of text files or a character vector.
Automatic chunking with configurable size and overlap.
RAG semantic search (edge_search()): Find the most relevant text
chunks for a query using cosine similarity over the embedding index.
RAG question answering (edge_ask()): Retrieval-augmented generation
that retrieves relevant context from an index and generates a grounded
answer. Supports custom system prompts and optional context return for
debugging/transparency.
Plumber API server (edge_serve()): Serve a model as a local
OpenAI-compatible REST API. Endpoints: /v1/completions,
/v1/chat/completions, /v1/embeddings, /v1/models, /health.
Supports optional API key authentication and CORS. Requires plumber.
Qwen3 model family in edge_list_models(): Added Qwen3-0.6B, 1.7B,
4B, and 8B pre-configured entries from the unsloth GGUF repository.
Friendly names in edge_download_model(): Now accepts model names from
edge_list_models() (e.g., edge_download_model("Qwen3-0.6B")) in addition
to HuggingFace repo IDs. Filename is auto-resolved from the model registry.
httr download fallback: .robust_download() now tries httr::GET before
R's download.file, improving reliability on corporate networks with custom
SSL certificates or proxy configurations.
SIMD optimization warning: On package load, warns if running without SIMD
(generic mode) and suggests reinstalling from source with
EDGEMODELR_SIMD=NATIVE for faster inference.
Fixed grammar-constrained generation failures (issue #41):
edge_grammar_completion(), edge_extract(), and edge_extract_batch() were
unusable due to two bugs. First, edge_json_grammar() emitted rule names like
field_1 containing underscores, which llama.cpp's grammar parser rejects
(only [a-zA-Z0-9-] is allowed in rule identifiers). Renamed to field-1.
Second, llama_sampler_accept() throws "Unexpected empty grammar stack" when
a token fully satisfies the grammar; the binding now catches this and
terminates cleanly, same as end-of-generation handling.
Fixed crash from silent context size override (issue #40 item 11):
Removed the auto-reduction of n_ctx for small models that silently changed
the user's requested context size. This caused segfaults when prompts exceeded
the reduced context. Context is now used as-is. Minimum n_ctx lowered from
512 to 128 for short-task use cases.
Fixed prompt echo in completion output (issue #40 item 1):
edge_completion() previously returned prompt + generated_text. Now returns
only the generated text, matching user expectations.
Added prompt length validation: All completion functions now validate that
the tokenized prompt fits within the model's context window before calling
llama_decode(). Exceeding the context now raises a clear R error instead of
crashing the process.
Model-native chat templates (issue #40 item 7): New
edge_chat_completion() function reads the model's chat template from GGUF
metadata (via llama_chat_apply_template) and formats messages correctly for
each model architecture (ChatML, Llama, Gemma, etc.). build_chat_prompt()
updated to accept an optional ctx parameter for native template formatting,
with ChatML as the generic fallback (replacing the old Human:/Assistant:
format).
edge_classify(ctx, text, c("positive", "negative", "neutral"))edge_extract(ctx, text, list(name = "string", role = "string"))CUDA GPU acceleration (Windows): New edge_install_cuda() and
edge_install_cuda_toolkit() functions set up GPU inference automatically.
edge_install_cuda() downloads the matching ggml-cuda dynamic backend
from llama.cpp releases and extracts the companion ggml-base.dll /
ggml.dll runtime libraries.edge_install_cuda_toolkit() copies nvcudart_hybrid64.dll from the
Windows DriverStore (already on any NVIDIA-driver machine, no download
required) and fetches cublas64 / cublasLt64 from NVIDIA's redistrib
server.edge_reload_cuda() activates the CUDA backend in the current R session
without restarting R.edge_cuda_info() reports whether CUDA is installed and active.n_gpu_layers = -1L to edge_load_model() for full GPU offload.Updated llama.cpp to build b8179 (GGML 0.9.7): Brings all upstream model architecture updates, sampler improvements, and quantization fixes.
std::regex to spend 40+
minutes in exponential backtracking. Added a hand-written fast path
unicode_regex_split_custom_qwen2() in unicode.cpp, matching the logic
of the existing llama-3 fast path. Qwen3-14B now loads in 0.3 s on CPU
(3.4 s on GPU including VRAM transfer). Covers QWEN2 and QWEN3.5 variants.abort() in ggml_abort() with raise(SIGABRT) under
#ifdef USING_R; replaces abort() token in ggml.cpp with
std::terminate().ggml_print_backtrace() body and fflush(stdout) /
fprintf(stderr, …) in ggml_abort() with #ifndef USING_R to remove
_Exit, stdout, and stderr symbol references from ggml.o on macOS.#define _GNU_SOURCE to ggml-cpu.c (required for SCHED_BATCH,
CPU_ZERO, pthread_setaffinity_np on Linux).CXX_STD = CXX17 replaces -std=c++17 in PKG_CXXFLAGS in both
Makevars and Makevars.win.-fno-builtin-printf added to GGML_CFLAGS to suppress
printf → puts optimizations.edge_install_cuda, edge_install_cuda_toolkit,
edge_reload_cuda, edge_cuda_info.Flash attention support: Enabled by default in edge_load_model() via flash_attn = TRUE. Reduces memory usage and improves attention computation speed on CPU.
Full hardware thread utilization: Removed the 4-thread cap for small contexts. edge_load_model() now uses all available CPU threads by default, with n_threads_batch set to max for prompt processing.
User-configurable threading: New n_threads parameter in edge_load_model() allows explicit control over CPU thread count. Pass NULL (default) for auto-detect or an integer to limit cores.
Apple Accelerate framework (macOS): Automatically links the Accelerate framework on macOS builds, enabling hardware-accelerated vDSP vector operations for faster matrix math.
Compiler auto-vectorization: Added -ftree-vectorize to GGML compilation flags on all platforms, allowing GCC/Clang to generate SIMD instructions for eligible loops beyond the hand-tuned GGML kernels.
SIMD-optimized build system: Replaced generic scalar fallback with architecture-aware SIMD detection in both Makevars (Unix) and Makevars.win (Windows)
User-configurable SIMD levels: Set EDGEMODELR_SIMD environment variable before install to select optimization level:
GENERIC: Scalar fallback (maximum compatibility)SSE42: SSE4.2 baseline (default on x86_64)AVX: AVX + F16C (Intel Sandy Bridge 2011+)AVX2: AVX2 + FMA + F16C (Intel Haswell 2013+, recommended)AVX512: AVX-512 (Intel Skylake-X 2017+)NATIVE: Uses -march=native for maximum performance on the build machineedge_simd_info(): New function to query compile-time SIMD status including architecture, compiler features, and GGML optimization flags
x86 architecture-specific quantization: Enabled optimized x86 quantization kernels (arch/x86/quants.c, arch/x86/repack.cpp) with SIMD-accelerated dot products and matrix operations
Fixed donttest examples: Changed resource-intensive examples from \donttest{} to \dontrun{} to prevent downloading multi-GB models during CRAN checks
Fixed M1 Mac compiler warnings: Added explicit static_cast<> for:
double to float conversions for temperature/top_p parameterssize_type to int32_t conversions for buffer size parametersFixed connection handling: Replaced on.exit() with tryCatch/finally for proper connection cleanup in loops (thanks @eddelbuettel)
Small Model Configuration Helper: New edge_small_model_config() function provides optimized settings for small models (1B-3B parameters)
Adaptive Batch Processing: Intelligent batch size optimization based on context length
Smart Thread Allocation: Context-aware CPU thread management
Automatic Context Optimization: Model size-based context tuning
Small Model Optimization Example: Comprehensive example demonstrating all optimization features
Enhanced Testing: New test suite for small model configuration
edge_find_ollama_models() - Discover all locally available Ollama models across platforms (Windows, macOS, Linux)edge_load_ollama_model() - Load Ollama models using convenient SHA-256 hash prefixes instead of full file pathstest_ollama_model_compatibility() - Built-in compatibility testing for Ollama modelsstd::filesystem on macOS builds<mach-o/dyld.h> inclusion with direct function declarations to avoid enum conflicts-march=native, -mtune=native, etc.) from Makevars for CRAN compatibilityedge_clean_cache() functionedge_load_model() - Load GGUF model files for inferenceedge_completion() - Generate text completionsedge_stream_completion() - Stream text generation with real-time callbacksedge_chat_stream() - Interactive chat session with streaming responsesedge_free_model() - Memory management and cleanupis_valid_model() - Model context validationedge_list_models() - List pre-configured popular modelsedge_download_model() - Download models from Hugging Face Hubedge_quick_setup() - One-line model download and setupThis release provides a complete, production-ready solution for Local Large Language Model Inference Engine in R, enabling private, offline text generation workflows.