4f07aefc97
Merge branch 'master' of git://factorcode.org/git/factor
...
Conflicts:
core/classes/tuple/tuple.factor
2008-03-31 19:42:21 -06:00
aa40350aa7
replace add* and add with prefix and suffix
2008-03-31 18:18:05 -06:00
72bfd57f30
Make ebnf forgiving of whitespace at end of expression
2008-04-01 11:28:14 +13:00
a41f8ef733
Mention how to fail from action in pegs
2008-03-31 17:26:42 +13:00
f66774e875
Add tests for semantic and add syntax for it to ebnf
...
Syntax is ?[ ...]?
For example:
[EBNF num=. ?[ number? ]? list=list:x num:y => [[ drop x y + ]] | num EBNF] { 1 2 3 4 5 6 } swap call .
2008-03-31 16:50:05 +13:00
c45eba6898
Add semantic parser
2008-03-31 16:34:59 +13:00
8a800361df
Merge commit 'origin/master'
2008-03-31 15:01:22 +13:00
729ac1d6dc
Some ebnf tweaks and tests to do with variables
2008-03-31 14:59:22 +13:00
ee2194d1dc
Allow variable names on elements
2008-03-31 14:03:16 +13:00
55a69392fa
First cut at variables in ebnf
2008-03-31 13:52:42 +13:00
5989680a7b
Ensure box parsers are never cached
2008-03-31 00:53:33 +13:00
bb8198d3d0
Declare stack effects for compiled parsers
2008-03-30 23:24:02 +13:00
78633e03a0
Allow var names in ebnf but ignore them for now
2008-03-30 19:01:47 +13:00
f9779f5c38
Fix conflict
2008-03-29 23:48:01 -05:00
a23e0ce15c
Fix hashcode* on parsers
2008-03-30 17:35:47 +13:00
d002e02948
Use left recursive grammar in peg.expr
2008-03-30 17:23:11 +13:00
af9e27823a
Add => action rule for an entire sequence
2008-03-30 17:17:31 +13:00
c22af5c7a6
Rename 2apply to bi@
2008-03-29 20:36:58 -05:00
691d26068d
Refactor peg compiler cache
...
Instead of a cache stored in a global variable, the compiled parser is stored in a slot in the parser delegate.
2008-03-30 14:00:45 +13:00
2aafe074b2
Merge branch 'master' of git://double.co.nz/git/factor
2008-03-29 00:38:31 -05:00
ca4f775756
Fix PEG:
2008-03-29 18:33:37 +13:00
8105e66aec
Add box parser to docs
2008-03-29 17:45:21 +13:00
86653e7a46
Don't use 'delay' parser in ebnf
2008-03-29 17:42:21 +13:00
2c3c66c6af
Update peg for words being moved
2008-03-28 23:37:52 -05:00
0db0d9cd44
Move towards having ebnf infer
2008-03-29 16:24:13 +13:00
5f37b4fc72
compiled pegs infer
2008-03-29 16:11:08 +13:00
7bf27a5eb2
EBNF test using Java Primary production
2008-03-29 03:41:40 +13:00
25eea7ea1b
Fix ebnf tests for left recursion
2008-03-29 02:51:49 +13:00
261539a86a
Unit test for left recursive grammar
2008-03-29 02:47:03 +13:00
dd979c8b3b
Indirect Left recursive grammars working
2008-03-29 02:45:21 +13:00
68cbdf76aa
Handle direct left recusion
2008-03-29 01:17:54 +13:00
010ce80076
Handle left recursion by failing again
2008-03-29 00:49:39 +13:00
cca4700e49
Fix ebnf for peg changes
2008-03-29 00:41:41 +13:00
4b353c7529
Rewrite peg internals
2008-03-29 00:37:40 +13:00
1d87e513f5
lr2 wip
2008-03-28 21:17:08 +13:00
d2190fd1ec
Direct left recurson working
2008-03-28 14:30:10 +13:00
749f10ba9f
Implement direct left recursion
...
As per VPRI Technical Report TR-2007-002 section 3.2
2008-03-28 14:30:10 +13:00
f596aa2d71
Handle compilation of circular parsers
2008-03-28 14:26:05 +13:00
f96a251f8a
Refactor pegs to remove MEMO: and use unique id's
2008-03-28 12:35:47 +13:00
89c7698738
Fix MEMO problem with seq* and choice*
2008-03-28 11:30:46 +13:00
fa8b311b27
Add packrat-parse, etc
2008-03-28 00:04:08 +13:00
f6b7f8197e
Add tests for left recusion in pegs
2008-03-27 23:54:34 +13:00
4e29081e93
Make left recursion in pegs a failed parse
...
Eventually left recursion will work fine, but this is prevents an infinite loop for now.
2008-03-27 18:15:22 +13:00
4c50daed22
Testcase for packrat behaviour
2008-03-27 17:58:08 +13:00
bc5f82255f
peg refactorings
2008-03-27 17:24:05 +13:00
708726d208
Add with-packrat word and more memoization
2008-03-27 15:50:27 +13:00
7c0535884e
Fix up peg memoization of compiled parsers
2008-03-27 15:34:49 +13:00
e6da3bc43a
Use cache in compiled-parser in peg
2008-03-27 14:55:14 +13:00
c0f4e37427
Fix usage of cache in pegs
2008-03-27 12:58:53 +13:00
2614792254
Implement packrat algorithm
2008-03-27 12:31:13 +13:00
c793a381fe
Add hook for packrat implementation
2008-03-27 11:24:06 +13:00
85a3ee3e5b
Remove memoization in 'compile' word in pegs
...
This creates issues when recompiling a an existing EBNF parser for reasons I've not yet tracked down. Disabling it slows things down but makes things work correctly till I investigate the issue.
2008-03-27 11:24:06 +13:00
de3e4e049f
Use cleave instead of get-slots in peg
2008-03-26 16:21:33 +13:00
1ec945ba4c
Use new slots in peg.ebnf
2008-03-26 16:16:23 +13:00
8569d18068
Use new slots in peg
2008-03-26 16:08:14 +13:00
b1561de0f6
Reduce amount of generated code for peg token parser
2008-03-26 15:40:17 +13:00
9b7246555a
Fix just parser in pegs
2008-03-26 15:26:54 +13:00
943b02ab2f
Fix performance regression in pegs
...
delay parser is improved to use a memoized quotation so the
construction and compilation of the parser at runtime only
occurs once.
Changed compile so it would use equality rather than identity
for memoization purposes.
2008-03-22 01:59:16 +13:00
d1e0aa6e80
Get peg subvocabs working again
2008-03-22 00:58:53 +13:00
a14854520d
Compile pegs down to words
2008-03-21 03:05:21 +13:00
1c6882b32c
Rip out packrat stuff
...
It was broken since the transition to generating compiled quotations. As far as I know, no one was using packrat-parse anyway.
Rework in progress...
2008-03-21 01:26:20 +13:00
d1e7ede35d
Add support for & syntax in ebnf
2008-03-20 17:25:27 +13:00
dbd0583044
Tidy up expr groups
2008-03-20 17:19:41 +13:00
8ade4f9b5b
Fix vocab name in expr tests
2008-03-20 17:16:50 +13:00
44954753bd
Change <EBNF .. EBNF> to [EBNF .. EBNF]
2008-03-20 17:13:27 +13:00
e7980ebc61
More refactoring of EBNF
...
<EBNF .. EBNF> now produces a quotation that when called does the parsing
EBNF: foo ... ;EBNF creates a 'foo' word with stack effect (string -- result)
when called it parses the string and returns the result.
2008-03-20 17:11:09 +13:00
7dc772db26
Refactor ebnf parser generation
2008-03-20 16:05:10 +13:00
c1f69f01be
Change ordering of [[ ... ]]
2008-03-20 14:57:12 +13:00
39c228db6d
Update peg.pl0 to use range pattern syntax
...
This allows removing the words for ident and number, replacing them
with EBNF expressions.
2008-03-20 14:22:14 +13:00
68388fbed9
Updated peg.expr to use range-pattern for digits
2008-03-20 14:15:06 +13:00
ec4f964e4f
Fix pl0 for EBNF syntax changes
2008-03-20 14:06:21 +13:00
795ef0ae3b
Add ranges to EBNF syntax
...
This works:
<EBNF letter = [a-zA-Z] EBNF>
and
<EBNF not-digit = [^0-9] EBNF>
2008-03-20 13:55:19 +13:00
264284d0c4
Add range-pattern parser
2008-03-20 13:40:22 +13:00
97b58580a7
Add expression evaluator example for EBNF
2008-03-20 03:30:53 +13:00
92d8140d87
Change ebnf-action to properly nest with attached parser
...
This allows removal of last-parser hack. Syntax of EBNF changes though.
Now an action must attach to a group:
<EBNF foo = (a b c) [[ ...act on group... ]] EBNF>
2008-03-20 03:05:40 +13:00
65fabeec11
remove => action and replace it with [[ code ]] in EBNF
...
Previously the action had to be a factor word and could only appear at the end of a rule:
: aword ( ast -- ast ) drop V{ 1 2 } ;
<EBNF foo = "a" "b" => aword EBNF>
Now actions can appear anywhere after an element, and can be any factor code between [[ ... ]] delimiters:
<EBNF foo = "a" "b" [[ drop V{ 1 2 } ]] EBNF>
<EBNF foo = "a" [[ drop 1 ]] "b" [[ drop 2 ]] EBNF>
Unfortunately since this means the ebnf>quot code uses the equivalent of eval, it no longer compiles nicely since it can't be inferred. The generated parsers however do compile.
2008-03-20 02:16:30 +13:00
c0b7bdf823
Add *, + and ? to list of non-allowed ebnf identifier characteres
2008-03-20 01:25:57 +13:00
82d54d3776
EBNF syntax change
...
[ ... ] is now ( ... )?
{ ... } is now ( ... )*
Added ( ... )+
2008-03-20 01:18:09 +13:00
64135b73e1
Add support for ensure-not and parsing any single character to EBNF
...
This allows, for example:
foo = {!("_" | "-") .}
This will match zero or more of any character, except for _ or -
2008-03-19 19:15:52 +13:00
208c88c449
Update pl0 for ebnf changes, and add more tests
2008-03-19 18:35:45 +13:00
eef6ae7827
Remove need for '.' to terminate rule lines in EBNF
2008-03-19 18:07:31 +13:00
9403d97e22
Add syntax-pack and grouped to ebnf refactoring
2008-03-19 17:52:22 +13:00
708d55fb8e
Add syntax word for ebnf
2008-03-19 17:37:08 +13:00
7578538122
Minor tidyup of ebnf
2008-03-19 17:34:47 +13:00
cc9a17b551
Use choice* and seq* in ebnf
2008-03-19 17:00:53 +13:00
4b37c9098e
Use multiline for parsing EBNF string
2008-03-19 16:54:42 +13:00
ef437c96da
add 1token and docs
2008-03-12 10:43:18 -05:00
5af2be3798
More documentation fixes
2008-03-11 21:01:39 -05:00
32526206f1
Help fixes
2008-03-11 19:51:58 -05:00
a336cb7570
Unit test fixes
2008-03-06 13:46:15 -06:00
f84761ae0c
fix docs for delay
2008-03-05 18:15:28 -06:00
47a96775d8
Add PEG: word
2008-03-03 16:57:30 -06:00
05a02ade7a
Unix <process-stream> now compiles
2008-03-03 16:45:18 -06:00
6378d38d63
add missing usings
2008-03-03 14:32:37 -06:00
2f48327b47
move non-core peg parsers to peg.parsers
...
document and unit test peg.parsers
add just parser
2008-03-03 13:28:53 -06:00
e98cd1fd59
New convention for unit tests
2008-03-01 17:00:45 -05:00
635b02ca27
implement list-of, list-of* in terms of (list-of)
...
add 2choice 3choice
2008-02-26 17:13:15 -06:00
11147c7bc0
add 2seq, 3seq
2008-02-26 15:17:17 -06:00
d165f29b36
Merge git://double.co.nz/git/factor
2008-02-25 16:32:36 -06:00
f6845d43d3
Massive name change in files, string streams
2008-02-15 22:20:31 -06:00
8542dc5812
Add seq* and choice* to peg
2008-02-14 10:39:37 +13:00
2541c62e29
Fix code for math.parser changes
2008-02-06 21:15:47 -06:00
c977d4a7fd
Merge git://factorcode.org/git/factor
2008-02-02 18:47:15 +13:00
bff385269c
Lot's of USING: fixes for ascii or unicode
2008-02-01 18:26:32 -06:00
af13a47485
Adding tags
2008-01-31 23:04:11 -06:00
f272d54e49
Allow dashes in non-terminal names
2008-01-18 15:45:08 +13:00
2d79bdb09a
Fix peg.search for recent factor changes
2008-01-17 11:08:13 +13:00
0f3160cc5d
Merge commit 'chris/master'
2008-01-17 11:05:38 +13:00
240217f39e
Fix peg issues with recent factor changes
2008-01-13 04:38:59 -05:00
8285eeda9a
Fix conflict
2008-01-06 13:30:23 -04:00
e7cf83a57a
First attempt at compiling peg parsers to quotations
2007-12-21 13:16:14 +13:00
b655d4c759
Add tags to peg, peg.ebnf and peg.pl0
2007-12-20 17:11:37 +13:00
a376cb6cc3
Add summary, author and tags to peg.search
2007-12-20 17:10:45 +13:00
6a8bfad475
Rename peg.replace to peg.search
2007-12-20 17:09:12 +13:00
6bd283ffa3
Convert parser-combinators.replace to pegs
2007-12-20 17:06:21 +13:00
d45ed669f8
Add digit, number and string parser to peg
2007-12-20 11:54:17 +13:00
d7217801c3
Documentation fixes
2007-12-11 22:36:40 -05:00
38beaac720
Infinite left recursion now causes a failed parser rather than a call stack error
2007-11-30 00:01:03 +13:00
3003911e07
Make some parsers uses MEMO:
2007-11-29 23:49:34 +13:00
362f2d3436
Fix packrat caching issue
2007-11-29 23:42:46 +13:00
b51e4f642e
Fix broken ebnf actions
2007-11-29 17:41:58 +13:00
a4461ae408
Tidy up ebnf compilation
2007-11-29 17:24:02 +13:00
f94c280e06
Fix pl0 tests
2007-11-29 12:49:51 +13:00
89bbd21362
Add packrat caching to peg
2007-11-29 12:35:45 +13:00
d3ac10aefc
Redo PL/0 parser using ebnf
2007-11-28 16:07:23 +13:00
e0adc1a7fa
Add optional to ebnf
2007-11-28 15:49:14 +13:00
d68a78c4a6
fix grouping of repeat0 in ebnf
2007-11-28 15:32:04 +13:00
3372ad8f68
Fix some peg breakage
2007-11-28 15:27:35 +13:00
0ef96c87d9
Add grouping operators for ebnf
2007-11-28 15:26:25 +13:00
15b1533f20
Fix precedence between choice/sequence in ebnf
2007-11-28 15:14:11 +13:00
28e9c0e6e0
Fix ebnf tests
2007-11-28 13:05:53 +13:00
35f96d1c85
Use words instead of quotations in ebnf actions
2007-11-28 13:03:16 +13:00
e5e430be4f
Remove ebnf debug
2007-11-28 12:52:05 +13:00
c455336da6
Add action rule to ebnf
2007-11-28 12:50:04 +13:00
4f0c40c05a
Change EBNF: to <EBNF ... EBNF>
2007-11-28 11:52:17 +13:00
88e93446b2
Add EBNF: word
2007-11-28 11:46:06 +13:00
7a414869de
Support for repeat0 in ebnf
2007-11-28 11:33:21 +13:00
16a0cc9eb1
add ebnf>quot
2007-11-28 11:25:34 +13:00
38806885e6
Compile ebnf->factor
2007-11-28 11:07:17 +13:00
ed359b6623
Syntax tree for ebnf
2007-11-28 10:45:21 +13:00
31d57422da
Start of EBNF parser
2007-11-27 18:13:36 +13:00
9f2f45cd71
More peg.pl0 additions
...
This parser is currently really ugly. The goal is to tidy up peg
so this parser looks more like the EBNF.
2007-11-27 16:45:32 +13:00
e49d84ce97
Add 'delay' parser to peg
2007-11-27 16:16:21 +13:00
ea2d4ea261
Work on PL/0 Grammar as a PEG example
2007-11-27 15:56:26 +13:00
5fb6af754b
Add hide combinator
2007-11-27 15:45:00 +13:00
055276ca25
Add 'sp' parser to skip whitespace
2007-11-27 15:36:26 +13:00
e6b6bb8a5c
Add satisfy parser in peg
2007-11-27 15:08:16 +13:00
1eed006a29
Add author information to peg
2007-11-27 12:36:14 +13:00
99b39e0351
Some help for pegs
2007-11-27 12:22:33 +13:00
6476eb765e
remove parse-state from peg
2007-11-27 11:57:08 +13:00
167f2d716d
add action parser
2007-11-22 11:19:16 +13:00
2a464ea2c6
add ensure-not parser
2007-11-22 11:19:16 +13:00
129f68d428
add ensure parser
2007-11-22 11:19:16 +13:00
ffa71ef86f
add optional parser
2007-11-22 11:19:16 +13:00
e9df13dad5
remove match from parse results
2007-11-22 11:19:16 +13:00
691c62501f
add repeat0 and repeat1
2007-11-22 11:19:16 +13:00
2d3fe08403
Add choice parser
2007-11-22 11:19:15 +13:00
aacf88a72d
First cut at peg style packrat parser
2007-11-22 11:19:15 +13:00
Eduardo Cavazos
Chris Double
Slava Pestov
Doug Coleman
Slava Pestov
dharmatech
Doug Coleman
Slava Pestov
Daniel Ehrenberg
Chris Double