XhdZddlmZmZmZmZmZddlmZddl m Z ddl m Z ddl mZddlmZd d lmZd d lmZerdd lmZmZGd deZy)aThis module implements an Earley parser with a dynamic lexer The core Earley algorithm used here is based on Elizabeth Scott's implementation, here: https://www.sciencedirect.com/science/article/pii/S1571066108001497 That is probably the best reference for understanding the algorithm here. The Earley parser outputs an SPPF-tree as per that document. The SPPF tree format is better documented here: http://www.bramvandersanden.com/post/2014/06/shared-packed-parse-forest/ Instead of running a lexer beforehand, or using a costy char-by-char method, this parser uses regular expressions by necessity, achieving high-performance while maintaining all of Earley's power in parsing any CFG. ) TYPE_CHECKINGCallableOptionalListAny) defaultdict)Tree)UnexpectedCharacters)Token)Terminal)Parser) TokenNode) LexerConf ParserConfc\eZdZdddedfdddddeded ed ed eeeege fd efd Z ddZ y)rTF lexer_confr parser_confr term_matcherresolve_ambiguity complete_lexdebug tree_class ordered_setsc tj|||||||||jD cgc] } t| c} |_||_ycc} wN) BaseParser__init__ignorer r) selfrrrrrrrrts H/var/www/html/myenv/lib/python3.12/site-packages/lark/parsers/xearley.pyrzParser.__init__sM D*kqx{> (?sA Nc n   f d}ttj jj ig d d d}D]6}j || |||}|dk(r dz d n dz |dz }8j || |t dz k(sJ|S)Nc  i}j|D]}|j|}|st|jj|j d|}|j j |||fjs|j d}tdt|D]v}|j|d| }|st|jj|j d|}||j zj |||fx jD]}||}|s|j j|Dcgc]}||df c}|j j|Dcgc]$}|js|jk(s ||df&c}j} j} j | j i|dzD]\}} } | Չ| _dz| _|dz| _|j#} | j| j$|dzf}t'| | j(}||vr||n|j+|j,|| _| j.j1| j|j2| j$|j.|n|} | jj4vr| j7|  | j7| |dz=| sisg| set9t;|d}t=||Dchc]}|jjc}t?|tAd|D|| Scc}wcc}wcc}w)a|The core Earley Scanner. This is a custom implementation of the scanner that uses the Lark lexer to match tokens. The scan list is built by the Earley predictor, based on the previously completed tokens. This ensures that at each phase of the parse we have a custom lexer context, allowing for more complex ambiguities.rrNcB|jjjSr)ruleoriginnamekeys r#z-Parser._parse..scan..|sH\H\r*c34K|]}|jywr)s).0is r# z.Parser._parse..scan..~sH^QRH^s)stateconsidered_rules)!Setexpectr r)groupendappendrrangelenr extend is_completer/end_line end_columnend_posadvancestartrtype setdefault SymbolNodenode add_familyr' TERMINALSaddlistsortedr set frozenset)r1to_scan node_cacheitemmr"r/jx next_to_scannext_setrBtokennew_itemlabel token_noder4columnsdelayed_matchesmatchr! start_symbolstream terminals text_column text_line transitivess r#scanzParser._parse..scan)sJ) R$++vq1dkk.. Ay+VA#AEEG,33dAq\C((GGAJ!&q#a&!1RA %dkk1Sqb6 :A $)$++*:*:AGGAJ9Va$b /!%%' : A AD!Q< Q R R.[[ M!VQ'#AEEG,33QX4ZdAt_4Z[$AEEG,33QXYZQ[5L_c_o_otxtztzKuKdAt_5LM M 88:LxxzH NN8 $   r "'6ac&: +"eU$%.EN'2QE$$%EEM#||~H%ZZQ?E!*5)EJJ2G!HJ9>*9LJu$5R\RgRghmo~oso~o~AFpGSHHMMM,,XZZHNNTXT]T]_ij#H??dnn4 $$X.LL*' +* !$OL#'w<\(]#^ *61ilsNtdht{{O_O_Nt+.wrxs< @?#-($.GZGr-