Skip to main content

module analysis::grammars::dramb::Regression

rascal-0.40.17
drambiguity-0.3.5

Utility to run stored tests against a new grammar and report new issues.

Usage​

import analysis::grammars::dramb::Regression;

Source code​

http://github.com/cwi-swat/drambiguity/blob/main/src/analysis/grammars/dramb/Regression.rsc

Dependencies​

import analysis::grammars::dramb::Util;
import analysis::grammars::dramb::Model;
import ValueIO;
import IO;
import Exception;
import util::Maybe;
import ParseTree;
import List;

data ParsingRegression​

Models the bad things that can happen when we parse old strings with a new grammar.

data ParsingRegression  
     = error(str sentence, Symbol nonTerminal, loc src)
     | ambiguity(str sentence, Symbol nonTerminal, Tree cluster)
     | crash(str sentence, Symbol nonTerminal, value exception)
     ;

function testForParsingRegressions​

Run the stored regression test sentences against a new grammar.

bool testForParsingRegressions(type[&T <: Tree] newGrammar, loc oldModelLocation)

This function runs the stashed sentences in the model against a new version of the grammar. It returns true if no new errors are detected, and false if there are.

If this function returns false then it's time to run Dr Ambiguity interactively to:

  • either fix the examples to reflect to the new grammar (for example the syntax has changed)
  • or fix the grammar to make sure no new ambiguities or errors are introduced anymore.

Benefits​

  • Use this as a test function in your own syntax project, and run it during continuous integration testing.
info

It's best to use IO::findResources to locate the binary file that stores your drAmbiguity model.

Pitfalls​

  • This function does not provide the diagnostics features of Dr Ambiguity. It's better to use the interactive tool for that.
  • Storing binary files in git repositories is not recommended, but it is still the best way to keep a drAmbiguity model around.

function runRegressionTests​

Reads the old model from a binary file and then continues to run the regression tests.

list[ParsingRegression] runRegressionTests(type[&T <: Tree] newGrammar, loc oldModelLocation)

function runRegressionTests​

This is the workhorse that runs the regression tests; it simulates exactly what the interactive tool also does when a grammar is updated.

list[ParsingRegression] runRegressionTests(type[&T <: Tree] newGrammar, Model m)

By reparsing all the examples in the stored model we find out if there are new parse errors, new ambiguities, or new crashes. We collect the errors and return them as a list for later inspection. However, it is recommended to use the interactive Dr Ambiguity tool for diagnostics rather than studying this list.