o

�hg�@s\dZd
dl
Z
d
dlmZ
d
dlmZ
d
dlmZmZ
d
dl	m
Z

Gdd�d�Zdd	d
�
ZdS)z	SQL Lexer�N)
�Lock)
�
TextIOBase)�tokens�keywords)
�consumec@sXeZ
dZd
ZdZe�Zedd��
Zdd�Z	dd�Z
d	d
�Zdd�Zd
d�Z
ddd�
ZdS)�LexerzrThe Lexer supports configurable syntax.
    To add support for additional keywords, use the `add_keywords` method.Nc

Cs^|j�!
|j
d
ur|�|_
|j
��
Wd
�
|j
SWd
�
|j
S1s'w



Y
|j
S)zRReturns the lexer instance used internally
        by the sqlparse core functions.N)�_lock�_default_instance�default_initialization)
�cls�r�A/usr/local/CyberCP/lib/python3.10/site-packages/sqlparse/lexer.py�get_default_instance0s




��
��zLexer.get_default_instancec

Cs�|��
|�
tj�

|�tj�

|�tj�

|�tj�

|�tj�

|�tj	�

|�tj
�

|�tj�

|�tj�

|�tj
�

d
S)zlInitialize the lexer with default dictionaries.
        Useful if you need to revert custom syntax settings.N)�clear�
set_SQL_REGEXr�	SQL_REGEX�add_keywords�KEYWORDS_COMMON�KEYWORDS_ORACLE�KEYWORDS_MYSQL�KEYWORDS_PLPGSQL�KEYWORDS_HQL�KEYWORDS_MSACCESS�KEYWORDS_SNOWFLAKE�KEYWORDS_BIGQUERY�KEYWORDS�
�selfrrr
r
:s









zLexer.default_initializationc

Csg|_g|_
d
S)z�Clear all syntax configurations.
        Useful if you want to load a reduced set of syntax configurations.
        After this call, regexps and keyword dictionaries need to be loaded
        to make the lexer functional again.N)�
_SQL_REGEX�	_keywordsrrrr
rIs

zLexer.clearcs$tj
tjB��f
d
d�|
D�
|_dS)z.Set the list of regex that will parse the SQL.c
s"g|]
\}
}t�
|
��j|f�qSr)�re�compile�match)�.0�rx�tt�
�FLAGSrr
�
<listcomp>Ts��z'Lexer.set_SQL_REGEX.<locals>.<listcomp>N)r �
IGNORECASE�UNICODEr)rrrr&r
rQs

�zLexer.set_SQL_REGEXcCs|j�
|
�

d
S)zhAdd keyword dictionaries. Keywords are looked up in the same order
        that dictionaries were added.N)r�append)rrrrr
rYszLexer.add_keywordscCs6|
��}|j
D]}||vr|||
f
Sqtj|
fS)
z�Checks for a keyword.

        If the given value is in one of the KEYWORDS_* dictionary
        it's considered a keyword. Otherwise, tokens.Name is returned.
        )�upperrr�Name)r�value�val�kwdictrrr
�
is_keyword^s



�
zLexer.is_keywordc	cs
�t|
t
�r
|
��}
t|
t�rn,t|
t�r3|r|
�|�
}
nz|
�d
�
}
Wnty2


|
�d�
}
Yn
wtd�t	|
�
�
�
�
t
|
�
}|D]A\}}|jD]3\}}||
|�}|sUqIt|tj
�rc||��fV
n
|tjurp|�|���
V
t||��|d�

ntj|fV
qBdS)a�

        Return an iterable of (tokentype, value) pairs generated from
        `text`. If `unfiltered` is set to `True`, the filtering mechanism
        is bypassed even if filters are defined.

        Also preprocess the text, i.e. expand tabs and strip it if
        wanted and applies registered filters.

        Split ``text`` into (tokentype, text) pairs.

        ``stack`` is the initial stack (default: ``['root']``)
        zutf-8zunicode-escapez+Expected text or file-like object, got {!r}�
N)�
isinstancer�read�str�bytes�decode�UnicodeDecodeError�	TypeError�format�type�	enumeraterr�
_TokenType�groupr�PROCESS_AS_KEYWORDr1r�end�Error)	r�text�encoding�iterable�pos�char�rexmatch�action�
mrrr
�
get_tokensks>�











�

�










��zLexer.get_tokens�
N)�__name__�
__module__�__qualname__�__doc__r	rr�classmethodrr
rrrr1rJrrrr
rs



	
rcCst�
��||
�S)
z�Tokenize sql.

    Tokenize *sql* using the :class:`Lexer` and return a 2-tuple stream
    of ``(token type, value)`` items.
    )rrrJ)�sqlrCrrr
�tokenize�srRrK)
rOr �	threadingr�ior�sqlparserr�sqlparse.utilsrrrRrrrr
�<module>
s