????
Your IP : 216.73.217.84
�
�j� � �� � d Z ddlZddlmZ ddlmZmZmZmZ ddl Z ddl m
Z
m Z m Z ddl
mZmZ e G d� d� � � � Zd ee fd
�Zd � Zd eeee f fd �ZdS )
zF
Represents storage where ssa data is collected, stored and extracted
� N)� dataclass)�List�Iterator�Tuple�Dict)�func�cast�distinct)�
session_scope�
RequestResultc �D � e Zd ZU eed< ee ed< eed< eed< dS )�
DomainData� domain_name�domain_total_reqs�is_a_wordpress_domain� urls_numberN)�__name__�
__module__� __qualname__�str�__annotations__r �int�bool� � �J/opt/cloudlinux/venv/lib64/python3.11/site-packages/ssa/modules/storage.pyr r sF � � � � � � ������C�y� � � � � � � ������r r �returnc # �z K � t | � � 5 }|� t j t j dt j � � t j t j � � t j t j
� � t j t t j
� � � � � � � t j t j dt j � � � � � t j t j dt j � � � � }t! j |d� �� � }|D ]^\ }}t% |� � }d}dgdz }|D ]*\ } }
} } }
| |t' |
� � < t ||
� � }�+t) ||| |�� � V � �_ ddd� � dS # 1 swxY w Y dS )z7
Iterates data from database domain-by-domain.
�%Hc � � | d S �Nr r ��items r �<lambda>z#iter_domains_data.<locals>.<lambda>/ s
� �VZ�[\�V]� r )�keyr � )r r r r N)r �queryr �domainr �strftime�
created_at�Count�id�max� wordpress�countr
�path�group_by�order_by� itertools�groupby�tupler r )�engine�db�results_by_hour�results_by_hour_groupedr �group�domain_results_by_hourr �requests_number_by_hour�_�hour� requests_num� is_wordpress�urlss r �iter_domains_datarB s� � � � �
�v� � � !�"� �(�(� � � �M�$�
� 8� 9� 9� �J�}�'� (� (� �H�]�,� -� -� �J�x�
� 2�3�3� 4� 4�
�
� �(� � � �M�$�
� 8� 9� 9�
�
� �(� � � �M�$�
� 8� 9� 9�
�
� � #,�"3�O�I]�I]�"^�"^�"^� �"9� � � �K��%*�5�\�\� "��K� ()�c�B�h� #�=S�
5�
5�9��4� �|�T�5A�'��D� � �2� !�+�t� 4� 4� � � � '�"9�&2� '� � � �
�
�
�
� �#!� !� !� !� !� !� !� !� !� !� !� !���� !� !� !� !� !� !s �FF0�0F4�7F4c
# � K � t | � � 5 }|� t j t j dt j � � t j t t j t j � � � � � d� � t j
t j � � � d� � t j t t j t j � � � � � d� � � � � t j |k � � � t j � |� � � � � t j t j dt j � � � � � t j t j dt j � � � � }d}dgdz dgdz dgdz }}}|D ]q\ } }
} } }
|r/|| k r)|t+ ||||�� � fV � dgdz dgdz dgdz }}}| |t- |
� � < | |t- |
� � < |
|t- |
� � < | }�r| t+ | |||�� � fV � ddd� � dS # 1 swxY w Y dS ) z6
Iterates urls data from database url-by-url.
r �url_throttled_reqs�url_total_reqs�
url_slow_reqsNr r& )r0 rD rE rF )r r' r r0 r r) r* �Sumr �hitting_limits�
sqlalchemy�Integer�labelr+ r, �is_slow_request�filterr( �in_r1 r2 �dictr )r6 r � all_pathsr7 � urls_data�
previous_pathrD rE rF r0 r> �url_throttled_req�
url_total_req� url_slow_reqs r �iter_urls_datarV C s� � � � �
�v� � � 1
�"��H�H� � � �M�$�
� 8� 9� 9� �H�T� �,�j�.@�� �
�
��u�)�*�*� �J� � �
�
��e�$�%�%� �H�T� �-�z�/A�C� C�
�
��e�O�$�$�
�
� �&� � �K� /�
�
�
�&� � � "� "�9� -� -�
�
�
�(� � ��
�d�M�4L� M� M�
�
�
�(� � ��
�d�M�4L� M� M�
�
�% �, �
�
�C�"�H�q�c�B�h���b�� -:�N�� KT� !� !� F�D�$�)�=�,� �
1� �$�!6�!6�#�T�&�'9�#1�"/� &� &� &� � � � � �C�"�H�q�c�B�h���b�� 5B�N�"� ->� �s�4�y�y� )�(5�N�3�t�9�9� %�'3�M�#�d�)�)� $� �M�M��D�� 1�)�'�
�
�
�
�
�
�
�Y1
� 1
� 1
� 1
� 1
� 1
� 1
� 1
� 1
� 1
� 1
� 1
���� 1
� 1
� 1
� 1
� 1
� 1
s �I$J�J�
Jc # � K � t | � � 5 }|� t j t j � � � t j |k � � � t j � � }t j |d� � � }|D ]\ }}|d� |D � � fV � � ddd� � dS # 1 swxY w Y dS )zA
Get information about durations of requests url-by-url.
c � � | d S r! r r" s r r$ z#get_url_durations.<locals>.<lambda>� s
� �D��G� r c � � g | ]\ }}|��S r r )�.0r= �durations r �
<listcomp>z%get_url_durations.<locals>.<listcomp>� s � �:�:�:�[�Q���:�:�:r N)
r r' r r0 r[ rM r( r2 r3 r4 )r6 r r7 rQ �durations_by_pathr% r: s r �get_url_durationsr^ { s# � � � �
�v� � � ;�"��H�H� � � � "�
�
� �&� � �K� /�
�
�
�(� � �
�
� � &� -� �+�+� -� -��+� ;� ;�J�C���:�:�E�:�:�:�:� :� :� :� :� ;�;� ;� ;� ;� ;� ;� ;� ;� ;� ;� ;� ;���� ;� ;� ;� ;� ;� ;s �BB7�7B;�>B;)�__doc__r3 � dataclassesr �typingr r r r rI r r r
�ssa.dbr r r rB rV r r r^ r r r �<module>rc s, ��� � � � � � !� !� !� !� !� !� .� .� .� .� .� .� .� .� .� .� .� .� � � � � +� +� +� +� +� +� +� +� +� +� /� /� /� /� /� /� /� /� �� � � � � � � ���%��*�!5� %� %� %� %�N5
� 5
� 5
�p;�d�3��c�
�?�.C� ;� ;� ;� ;� ;� ;r