Prompting Effects on a Quantized Code LLM

Three empirical studies of DeepSeek Coder 6.7B (base and instruct, 4-bit NF4 quantization, greedy decoding) on a fixed 20-problem HumanEval subset:

1. Code generation — base vs instruct completions, scored with the HumanEval harness before and after post-processing.
2. Code reasoning — output prediction and pytest test generation, comparing a minimal "vanilla" prompt against a "crafted" prompt with chain-of-thought and few-shot examples.
3. Cross-language work — Python-to-Java translation (HumanEval-X, scored by compiling and running the Java tests) and bug classification (HumanEvalPack buggy solutions).

The short version of what we found: almost all usable generation quality comes from instruction tuning plus mechanical post-processing (raw pass@1 is 5% for both variants; trimming to the first function definition takes instruct to 45%). Crafted prompting buys large gains on reasoning-shaped tasks (+25 to +30 points on prediction, translation, and bug classification) — but on test generation, coverage-obsessed instructions produced more tests with lower correctness and more unusable suites. Asking for more is not free.

Results

Code generation (20-problem HumanEval subset, greedy decoding):

DeepSeek Coder 6.7B	Model output	pass@1
base	raw	5% (1/20)
base	post-processed	25% (5/20)
instruct	raw	5% (1/20)
instruct	post-processed	45% (9/20)

Prompt engineering effects (vanilla vs crafted prompt, same model and tasks):

Task (instruct model)	Vanilla prompt	Crafted prompt
Output prediction	35% (7/20)	60% (12/20)
Python -> Java translation	45% (9/20)	75% (15/20)
Bug classification	0% (0/20)	30% (6/20)

Generated test suites (pytest + line coverage on the same 20 problems):

Test generation	Usable suites	Mean line coverage	Suites at 100%
vanilla prompt	19/20	98.1%	16
crafted prompt	16/20	95.8%	14

Tables generated from the committed run artifacts by make tables (scripts/summarize_results.py); CI fails on drift.

Full per-task tables: results/RESULTS.md.

Context from the accompanying project report (numbers not derivable from the committed artifacts are marked as such):

• Output prediction scoring. The tables above are scored by the run's answer parser. A manual review (from the project report) found the parser had marked five answers wrong that were actually correct; after that review the rates are vanilla 9/20 (45%) and crafted 15/20 (75%). The committed artifacts were deliberately left unmodified, so the generated tables report the stricter parser-scored numbers.
• Generated-test correctness. Of the tests generated, 29/57 passed under the vanilla prompt (~51%) vs 32/79 under the crafted prompt (~40.5%) (from the project report; per-test outcomes were not retained as artifacts). The crafted prompt generated more tests and broader edge-case probing, at the cost of correctness.
• Reasoning-to-tests link. Per-task correlation between output-prediction success and generated-test pass rate was r ≈ 0.37 under vanilla prompting and r ≈ 0.00 under crafted prompting (from the project report) — the complexity of the coverage-focused prompt washed out the reasoning signal.
• Translation failure mix. Crafted prompting shifted translation failures from mostly compilation errors (8 of 11 vanilla failures) to mostly logic/test failures (4 of 5 crafted failures): the explicit type-conversion mappings moved the model from struggling with syntax to struggling with semantics.

Method

• Model: deepseek-ai/deepseek-coder-6.7b-base and -instruct, loaded 4-bit (NF4, double quantization, bfloat16 compute) via bitsandbytes; greedy decoding throughout.
• Tasks: a fixed 20-problem subset of HumanEval, selected once with a deterministic seeded sampler (experiments/01_generation/dataset_generation.py) and reused across all studies — the HumanEval-X (Python/Java) and HumanEvalPack subsets carry the same task IDs, so every study observes the same problems.
• Prompt variants: "vanilla" uses the model's stock assistant preamble and a bare instruction; "crafted" adds task-specific structure — step-by-step reasoning instructions, validated input/output examples (prediction), explicit Python-to-Java type-conversion mappings and the expected method declaration (translation), and the full problem specification plus a fixed reasoning sequence (bug classification).
• Scoring: functional correctness only. Generation runs through the HumanEval execution harness; translations must compile (javac) and pass the HumanEval-X Java tests; generated pytest suites are executed with line coverage recorded per suite (experiments/02_prediction_testgen/Coverage/).

Repository layout

experiments/
  01_generation/         Study 1: base vs instruct code generation
                         (subset selection, prompting + post-processing,
                          completions and harness verdicts as JSONL)
  02_prediction_testgen/ Study 2: output prediction and test generation
                         (Tests/ = model-generated pytest suites,
                          Coverage/ = per-suite pytest-cov reports)
  03_translation_bugs/   Study 3: Py->Java translation and bug classification
                         (HumanEval-X / HumanEvalPack subset builders + tasks)
scripts/                 summarize_results.py, make_figures.py
tests/                   CPU-only unit and artifact-integrity tests (what CI runs)
results/                 generated tables and figures (make tables / make figures)
docs/                    EXPERIMENTS.md - full GPU reproduction guide

Reproducing

Analysis path (no GPU, no downloads). The committed 20-problem subsets and run artifacts are the canonical record; everything reported above regenerates from them:

make setup    # venv + CPU-only analysis dependencies
make check    # lint + tests + regenerate tables/figures, fail on drift

make help lists all targets. The repo's own tests live in tests/; the model-generated suites under experiments/.../Tests/ are experiment artifacts and are never executed by CI.

Rerunning inference (GPU). The original runs used Colab T4/A100 instances. pip install -e ".[gpu]" installs the pinned experiment stack, and docs/EXPERIMENTS.md walks through each study end to end, including the HumanEval harness patches and the Java toolchain for translation scoring. Note: the subsets were selected with a fixed project seed that is deliberately not published, so a fresh dataset_generation.py --seed N run produces a different (equally valid) subset rather than reproducing the committed one.

Limitations and failure modes

• Scale: 20 problems per task, one greedy sample each — pass@1 point estimates with no variance bars. Treat differences of a problem or two as noise.
• Parser strictness: tagged-answer parsing in the prediction study created five false negatives (documented above); scoring protocols are part of the result.
• Coverage is not correctness: several generated suites hit 100% line coverage while asserting wrong expected values; the crafted prompt also produced more suites that never ran (4 vs 1), including three empty generations kept in-tree as failures.
• Hard translation ceiling: three problems failed in both prompt variants — Python eval(), Union return types, and multi-step string-state manipulation have no prompt-side fix.
• Shallow verdicts: of the six correct "buggy" classifications, only four had reasoning that matched the actual bug (from the project report).
• Rerun drift: different GPU/driver/library versions can shift individual outcomes; the committed artifacts are the record of the original runs.

Origin

This project originated as the three-part research project for UIUC's CS 598: Machine Learning for Software Engineering (Fall 2025), built with a four-person project team; this public copy is maintained by Sean Kraemer.

Data and licenses

Source	Used for	License
openai/openai_humaneval	generation, prediction, test generation	MIT
THUDM/humaneval-x	Python-to-Java translation	Apache-2.0
bigcode/humanevalpack	bug classification	MIT

Only 20-problem derived subsets and this project's run outputs are committed; the full datasets are downloaded by the dataset scripts. DeepSeek Coder weights are not redistributed here and remain subject to the DeepSeek license.

This repository is released under the MIT License.