From 2a09ebd5c184a6f80d0f4f85eda35e0dd55f706b Mon Sep 17 00:00:00 2001 From: Shuo Zhang Date: Wed, 20 Sep 2023 16:04:43 +0800 Subject: [PATCH 01/11] doc(readme): update readme, add 20B releasing info (#328) * fix(eval): StreamingDataset does not have an __len__ method. * doc(readme): update readme * update readme --- README-zh-Hans.md | 180 ++++++++++++++++++++++++----------- README.md | 171 +++++++++++++++++++++++---------- doc/imgs/modelscope_logo.png | Bin 0 -> 6186 bytes 3 files changed, 246 insertions(+), 105 deletions(-) create mode 100644 doc/imgs/modelscope_logo.png diff --git a/README-zh-Hans.md b/README-zh-Hans.md index 6679939..4ab9a84 100644 --- a/README-zh-Hans.md +++ b/README-zh-Hans.md @@ -33,26 +33,104 @@

- 👋 加入我们的推特、Discord 和 微信社区 + 👋 加入我们的 Discord 和 微信社区

## 简介 -InternLM ，即书生·浦语大模型，包含面向实用场景的70亿参数基础模型与对话模型 （InternLM-7B）。模型具有以下特点： +InternLM 是一个开源的轻量级训练框架，旨在支持大模型训练而无需大量的依赖。通过单一的代码库，它支持在拥有数千个 GPU 的大型集群上进行预训练，并在单个 GPU 上进行微调，同时实现了卓越的性能优化。在1024个 GPU 上训练时，InternLM 可以实现近90%的加速效率。 -- 使用上万亿高质量语料，建立模型超强知识体系； -- 支持8k语境窗口长度，实现更长输入与更强推理体验； -- 通用工具调用能力，支持用户灵活自助搭建流程； +基于InternLM训练框架，我们已经预训练了两个开源的预训练模型：InternLM-7B 和 InternLM-20B。 -提供了支持模型预训练的轻量级训练框架，无需安装大量依赖包，一套代码支持千卡预训练和单卡人类偏好对齐训练，同时实现了极致的性能优化，实现千卡训练下近90%加速效率。 +## 更新 -## 新闻 +[20230920] InternLM-20B 已发布，包括基础版和对话版。 +[20230822] InternLM-7B-Chat v1.1 已发布，增加了代码解释器和函数调用能力。您可以使用 [Lagent](https://github.com/InternLM/lagent) 进行尝试。 -我们开源了 InternLM-Chat-7B v1.1。该模型能够调用代码解释器和工具插件。你可以在 [Lagent](https://github.com/InternLM/lagent) 中体验这些新功能。 -## InternLM-7B +## Model Zoo -### 性能评测 +我们的模型在三个平台上发布：Transformers、ModelScope 和 OpenXLab。 + +| Model | Transformers | ModelScope | OpenXLab | | 发布日期 | +|---------------------------|------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|---|--------------| +| **InternLM Chat 20B** | [🤗internlm/internlm-chat-20b](https://huggingface.co/internlm/internlm-20b-chat) | [ Shanghai_AI_Laboratory/internlm-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b-chat/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-20b) | | 2023-09-20 | +| **InternLM 20B** | [🤗internlm/internlm-20b](https://huggingface.co/internlm/internlm-20b) | [ Shanghai_AI_Laboratory/internlm-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-20b) | | 2023-09-20 | +| **InternLM Chat 7B v1.1** | [🤗internlm/internlm-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | [ Shanghai_AI_Laboratory/internlm-chat-7b-v1_1](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-v1_1/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | | 2023-08-22 | +| **InternLM 7B** | [🤗internlm/internlm-7b](https://huggingface.co/internlm/internlm-7b) | [ Shanghai_AI_Laboratory/internlm-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | | 2023-07-06 | +| **InternLM Chat 7B** | [🤗internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) | [ Shanghai_AI_Laboratory/internlm-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | | 2023-07-06 | +| **InternLM Chat 7B 8k** | [🤗internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) | [ Shanghai_AI_Laboratory/internlm-chat-7b-8k](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-8k/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-8k) | | 2023-07-06 | + + +
+ InternLM-20B + +### 简介 +InternLM-20B 在超过 **2.3T** Tokens 包含高质量英文、中文和代码的数据上进行预训练，其中 Chat 版本还经过了 SFT 和 RLHF 训练，使其能够更好、更安全地满足用户的需求。 + +InternLM 20B 在模型结构上选择了深结构，InternLM-20B 的层数设定为60层，超过常规7B和13B模型所使用的32层或者40层。在参数受限的情况下，提高层数有利于提高模型的综合能力。此外，相较于InternLM-7B，InternLM-20B使用的预训练数据经过了更高质量的清洗，并补充了高知识密度和用于强化理解和推理能力的训练数据。因此，它在理解能力、推理能力、数学能力、编程能力等考验语言模型技术水平的方面都得到了显著提升。总体而言，InternLM-20B具有以下的特点： +- 优异的综合性能 +- 很强的工具调用功能 +- 支持16k语境长度（通过推理时外推） +- 更好的价值对齐 + +### 性能对比 + +在OpenCompass提出的5个能力维度上，InternLM-20B都取得很好的效果（粗体为13B-33B这个量级范围内，各项最佳成绩） + +| 能力维度 | Llama-13B | Llama2-13B | Baichuan2-13B | InternLM-20B | Llama-33B | Llama-65B | Llama2-70B | +|----------|-----------|------------|---------------|--------------|-----------|-----------|------------| +| 语言 | 42.5 | 47 | 47.5 | **55** | 44.6 | 47.1 | 51.6 | +| 知识 | 58.2 | 58.3 | 48.9 | 60.1 | **64** | 66 | 67.7 | +| 理解 | 45.5 | 50.9 | 58.1 | **67.3** | 50.6 | 54.2 | 60.8 | +| 推理 | 42.7 | 43.6 | 44.2 | **54.9** | 46.4 | 49.8 | 55 | +| 学科 | 37.3 | 45.2 | 51.8 | **62.5** | 47.4 | 49.7 | 57.3 | +| 总平均 | 43.8 | 47.3 | 49.4 | **59.2** | 48.9 | 51.9 | 57.4 | + +下表在一些有重要影响力的典型数据集上比较了主流开源模型的表现 + +| | 评测集 | Llama-13B | Llama2-13B | Baichuan2-13B | InternLM-20B | Llama-33B | Llama-65B | Llama2-70B | +|------|------------------|-----------|------------|---------------|--------------|-----------|-----------|------------| +| 学科 | MMLU | 47.73 | 54.99 | 59.55 | **62.05** | 58.73 | 63.71 | 69.75 | +| | C-Eval (val) | 31.83 | 41.4 | **59.01** | 58.8 | 37.47 | 40.36 | 50.13 | +| | AGI-Eval | 22.03 | 30.93 | 37.37 | **44.58** | 33.53 | 33.92 | 40.02 | +| 知识 | BoolQ | 78.75 | 82.42 | 67 | **87.46** | 84.43 | 86.61 | 87.74 | +| | TriviaQA | 52.47 | 59.36 | 46.61 | 57.26 | **66.24** | 69.79 | 70.71 | +| | NaturalQuestions | 20.17 | 24.85 | 16.32 | 25.15 | **30.89** | 33.41 | 34.16 | +| 理解 | CMRC | 9.26 | 31.59 | 29.85 | **68.78** | 14.17 | 34.73 | 43.74 | +| | CSL | 55 | 58.75 | 63.12 | **65.62** | 57.5 | 59.38 | 60 | +| | RACE (middle) | 53.41 | 63.02 | 68.94 | **86.35** | 64.55 | 72.35 | 81.55 | +| | RACE (high) | 47.63 | 58.86 | 67.18 | **83.28** | 62.61 | 68.01 | 79.93 | +| | XSum | 20.37 | 23.37 | 25.23 | **35.54** | 20.55 | 19.91 | 25.38 | +| 推理 | WinoGrande | 64.64 | 64.01 | 67.32 | **69.38** | 66.85 | 69.38 | 69.77 | +| | BBH | 37.93 | 45.62 | 48.98 | **52.51** | 49.98 | 58.38 | 64.91 | +| | GSM8K | 20.32 | 29.57 | **52.62** | **52.62** | 42.3 | 54.44 | 63.31 | +| | PIQA | 79.71 | 79.76 | 78.07 | 80.25 | **81.34** | 82.15 | 82.54 | +| 编程 | HumanEval | 14.02 | 18.9 | 17.07 | **25.61** | 17.68 | 18.9 | 26.22 | +| | MBPP | 20.6 | 26.8 | 30.8 | **35.6** | 28.4 | 33.6 | 39.6 | + +总体而言，InternLM-20B 在综合能力上全面领先于13B量级的开源模型，同时在推理评测集上接近甚至超越Llama-65B的性能。 + +- 评估结果来自 [OpenCompass 20230920](https://github.com/internLM/OpenCompass/)。 +- 由于 [OpenCompass](https://github.com/internLM/OpenCompass/) 的版本迭代，评估数据可能存在数值上的差异，所以请参考 [OpenCompass](https://github.com/internLM/OpenCompass/) 的最新评估结果。 + +
+ + +
+ InternLM-7B + +#### 模型更新 +[20230822] 通过使用更丰富的SFT类型数据，InternLM-7B-Chat v1.1模型支持代码解释和函数调用。模型结构与代码没有任何变化，因此可以使用与InternLM-7B-Chat完全一样的方式使用更强大的InternLM-7B-Chat v1.1。 + +#### 简介 +InternLM-7B 包含了一个拥有70亿参数的基础模型和一个为实际场景量身定制的对话模型。该模型具有以下特点： + +- 它利用数万亿的高质量令牌进行训练，建立了一个强大的知识库。 +- 它支持8k的上下文窗口长度，使得输入序列更长并增强了推理能力。 +- 它为用户提供了一个多功能的工具集，使用户能够灵活地构建自己的工作流程。 + +#### 性能对比 我们使用开源评测工具 [OpenCompass](https://github.com/internLM/OpenCompass/) 从学科综合能力、语言能力、知识能力、推理能力、理解能力五大能力维度对InternLM开展全面评测，部分评测结果如下表所示，欢迎访问[OpenCompass 榜单](https://opencompass.org.cn/rank)获取更多的评测结果。 @@ -72,27 +150,22 @@ InternLM ，即书生·浦语大模型，包含面向实用场景的70亿参数 - 以上评测结果基于 [OpenCompass 20230706](https://github.com/internLM/OpenCompass/) 获得（部分数据标注`*`代表数据来自原始论文），具体测试细节可参见 [OpenCompass](https://github.com/internLM/OpenCompass/) 中提供的配置文件。 - 评测数据会因 [OpenCompass](https://github.com/internLM/OpenCompass/) 的版本迭代而存在数值差异，请以 [OpenCompass](https://github.com/internLM/OpenCompass/) 最新版的评测结果为主。 -### Model Zoo -当前通过 InternLM 训练的 InternLM 7B 和 InternLM 7B Chat 已经开源，我们提供两种格式的模型权重以供使用。除了使用 Transformers 格式加载模型之外，还可以通过 InternLM 加载以下格式的权重直接进行继续预训练或人类偏好对齐训练 - -| 模型 | InternLM 格式权重下载地址 | Transformers 格式权重下载地址 | -| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------ | -| **InternLM 7B** | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | [🤗internlm/intern-7b](https://huggingface.co/internlm/internlm-7b) | -| **InternLM Chat 7B v1.1** | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | [🤗internlm/intern-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | -| **InternLM Chat 7B** | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | [🤗internlm/intern-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) -| **InternLM Chat 7B 8k** | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-8k) | [🤗internlm/intern-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) **局限性：** 尽管在训练过程中我们非常注重模型的安全性，尽力促使模型输出符合伦理和法律要求的文本，但受限于模型大小以及概率生成范式，模型可能会产生各种不符合预期的输出，例如回复内容包含偏见、歧视等有害内容，请勿传播这些内容。由于传播不良信息导致的任何后果，本项目不承担责任。 +
+ +## 使用案例 + ### 通过 Transformers 加载 -通过以下的代码加载 InternLM 7B Chat 模型 +通过以下的代码从 Transformers 加载 InternLM 模型 （可修改模型名称替换不同的模型） ```python >>> from transformers import AutoTokenizer, AutoModelForCausalLM ->>> tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-chat-7b-v1_1", trust_remote_code=True) ->>> model = AutoModelForCausalLM.from_pretrained("internlm/internlm-chat-7b-v1_1", trust_remote_code=True).cuda() +>>> tokenizer = AutoTokenizer.from_pretrained("internlm/internlm-chat-7b", trust_remote_code=True) +>>> model = AutoModelForCausalLM.from_pretrained("internlm/internlm-chat-7b", trust_remote_code=True).cuda() >>> model = model.eval() >>> response, history = model.chat(tokenizer, "你好", history=[]) >>> print(response) @@ -105,6 +178,24 @@ InternLM ，即书生·浦语大模型，包含面向实用场景的70亿参数 3. 集中注意力：避免分心，集中注意力完成任务。关闭社交媒体和电子邮件通知，专注于任务，这将帮助您更快地完成任务，并减少错误的可能性。 ``` +### 通过 ModelScope 加载 + +通过以下的代码从 ModelScope 加载 InternLM 模型 （可修改模型名称替换不同的模型） + +```python +from modelscope import snapshot_download, AutoTokenizer, AutoModelForCausalLM +import torch +model_dir = snapshot_download('Shanghai_AI_Laboratory/internlm-chat-7b-v1_1', revision='v1.0.0') +tokenizer = AutoTokenizer.from_pretrained(model_dir, device_map="auto", trust_remote_code=True,torch_dtype=torch.float16) +model = AutoModelForCausalLM.from_pretrained(model_dir,device_map="auto", trust_remote_code=True,torch_dtype=torch.float16) +model = model.eval() +response, history = model.chat(tokenizer, "hello", history=[]) +print(response) +response, history = model.chat(tokenizer, "please provide three suggestions about time management", history=history) +print(response) +``` + + ### 通过前端网页对话 可以通过以下代码启动一个前端的界面来与 InternLM Chat 7B 模型进行交互 @@ -123,44 +214,25 @@ streamlit run web_demo.py 我们使用 [LMDeploy](https://github.com/InternLM/LMDeploy) 完成 InternLM 的一键部署。 -```bash -python3 -m pip install lmdeploy -``` +1. 首先安装 LMDeploy: -执行以下命令，可以在终端与 `internlm-chat-7b` 模型进行交互式对话，或者通过 WebUI 与它聊天。 + ``` + python3 -m pip install lmdeploy + ``` -```bash -# 转换权重格式 -python3 -m lmdeploy.serve.turbomind.deploy internlm-chat-7b +2. 快速的部署命令如下： -# 在终端进行交互式对话 -python3 -m lmdeploy.turbomind.chat ./workspace + ``` + python3 -m lmdeploy.serve.turbomind.deploy InternLM-7B /path/to/internlm-7b/model hf + ``` -# 启动 gradio 服务 -python3 -m lmdeploy.serve.gradio.app ./workspace -``` -以上过程中，LMDeploy 使用的是 FP16 的计算精度。 +3. 在导出模型后，你可以直接通过如下命令启动服务一个服务并和部署后的模型对话 -除了 FP16 精度，LMDeploy 还支持 `internlm-chat-7b` 4bit 权重模型推理。它不仅把模型的显存减少到 6G，大约只有 FP16 的 40%，更重要的是，经过 kernel 层面的极致优化，其推理性能在 A100-80G 上可达到 FP16 的 2.4 倍以上。 - -以下是`internlm-chat-7b` 4bit 权重模型的部署方法。推理速度的 bechmark 请参考[这里](https://github.com/InternLM/lmdeploy/blob/main/docs/zh_cn/w4a16.md#%E6%8E%A8%E7%90%86%E9%80%9F%E5%BA%A6) - -```bash -# download prequnantized internlm-chat-7b model from huggingface -git-lfs install -git clone https://huggingface.co/lmdeploy/llama2-chat-7b-w4 - -# Convert the model's layout and store it in the default path, ./workspace. -python3 -m lmdeploy.serve.turbomind.deploy internlm-chat-7b ./llama2-chat-7b-w4 awq --group-size 128 - -# inference lmdeploy's turbomind engine -python3 -m lmdeploy.turbomind.chat ./workspace - -# serving with gradio -python3 -m lmdeploy.serve.gradio.app ./workspace -``` -LMDeploy 是涵盖了 LLM 任务的全套轻量化、部署和服务的工具箱。请参考 [部署教程](https://github.com/InternLM/LMDeploy) 了解 InternLM 的更多部署细节。 + ``` + python3 -m lmdeploy.serve.client {server_ip_addresss}:33337 + ``` +[LMDeploy](https://github.com/InternLM/LMDeploy) 支持了 InternLM 部署的完整流程，请参考 [部署教程](https://github.com/InternLM/LMDeploy) 了解 InternLM 的更多部署细节。 ## 微调&训练 diff --git a/README.md b/README.md index 0097aa8..f4b9351 100644 --- a/README.md +++ b/README.md @@ -33,26 +33,103 @@

- 👋 join us on Twitter, Discord and WeChat + 👋 join us on Discord and WeChat

## Introduction +InternLM is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies. With a single codebase, it supports pre-training on large-scale clusters with thousands of GPUs, and fine-tuning on a single GPU while achieving remarkable performance optimizations. InternLM achieves nearly 90% acceleration efficiency during training on 1024 GPUs. -InternLM has open-sourced a 7 billion parameter base model and a chat model tailored for practical scenarios. The model has the following characteristics: +Based on the InternLM training framework, we have pre-trained two open-sourced pretrained model InternLM-7B and InternLM-20B. + + +## News + +[20230920] InternLM-20B is released with base and chat versions. +[20230822] InternLM-7B-Chat v1.1 is released with code interpreter and function calling capability. You can try it with [Lagent](https://github.com/InternLM/lagent). + + +## Model Zoo + +Our models are released in three platforms: Transformers, ModelScope and OpenXLab. + +| Model | Transformers | ModelScope | OpenXLab | | Release Date | +|---------------------------|------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|---|--------------| +| **InternLM Chat 20B** | [🤗internlm/internlm-chat-20b](https://huggingface.co/internlm/internlm-20b-chat) | [ Shanghai_AI_Laboratory/internlm-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b-chat/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-20b) | | 2023-09-20 | +| **InternLM 20B** | [🤗internlm/internlm-20b](https://huggingface.co/internlm/internlm-20b) | [ Shanghai_AI_Laboratory/internlm-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-20b) | | 2023-09-20 | +| **InternLM Chat 7B v1.1** | [🤗internlm/internlm-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | [ Shanghai_AI_Laboratory/internlm-chat-7b-v1_1](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-v1_1/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | | 2023-08-22 | +| **InternLM 7B** | [🤗internlm/internlm-7b](https://huggingface.co/internlm/internlm-7b) | [ Shanghai_AI_Laboratory/internlm-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | | 2023-07-06 | +| **InternLM Chat 7B** | [🤗internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) | [ Shanghai_AI_Laboratory/internlm-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | | 2023-07-06 | +| **InternLM Chat 7B 8k** | [🤗internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) | [ Shanghai_AI_Laboratory/internlm-chat-7b-8k](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-8k/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-8k) | | 2023-07-06 | + +
+ InternLM-20B + +### Introduction +InternLM-20B was pre-trained on over **2.3T** Tokens containing high-quality English, Chinese, and code data. Additionally, the Chat version has undergone SFT and RLHF training, enabling it to better and more securely meet users' needs. + +In terms of model structure, InternLM-20B opted for a deeper architecture, with a depth set at 60 layers. This surpasses the conventional 7B and 13B models that utilize 32 or 40 layers. When parameters are limited, increasing the number of layers can enhance the model's overall capability. Furthermore, compared to InternLM-7B, the pre-training data used for InternLM-20B underwent higher quality cleansing and was supplemented with data rich in knowledge and designed for reinforcing understanding and reasoning capabilities. As a result, it exhibits significant improvements in understanding, reasoning, mathematical, and programming abilities—all of which test the technical proficiency of language models. Overall, InternLM-20B features the following characteristics: +- Outstanding overall performance +- Strong utility invocation capability +- Supports a 16k context length (Through inference extrapolation) +- Better value alignment. + +### Performace Evaluation + +On the 5 capability dimensions proposed by OpenCompass, InternLM-20B has achieved excellent results (the bolded scores represent the best performances within the 13B-33B parameter range). + +| Capability | Llama-13B | Llama2-13B | Baichuan2-13B | InternLM-20B | Llama-33B | Llama-65B | Llama2-70B | +|----------|-----------|------------|---------------|--------------|-----------|-----------|------------| +| Language | 42.5 | 47 | 47.5 | **55** | 44.6 | 47.1 | 51.6 | +| Knowledge | 58.2 | 58.3 | 48.9 | 60.1 | **64** | 66 | 67.7 | +| Understanding | 45.5 | 50.9 | 58.1 | **67.3** | 50.6 | 54.2 | 60.8 | +| Reasoning | 42.7 | 43.6 | 44.2 | **54.9** | 46.4 | 49.8 | 55 | +| Examination | 37.3 | 45.2 | 51.8 | **62.5** | 47.4 | 49.7 | 57.3 | +| Overall | 43.8 | 47.3 | 49.4 | **59.2** | 48.9 | 51.9 | 57.4 | + +The table below compares the performance of mainstream open-source models on some influential and typical datasets. + +| | Benchmarks | Llama-13B | Llama2-13B | Baichuan2-13B | InternLM-20B | Llama-33B | Llama-65B | Llama2-70B | +|------|------------------|-----------|------------|---------------|--------------|-----------|-----------|------------| +| Examination | MMLU | 47.73 | 54.99 | 59.55 | **62.05** | 58.73 | 63.71 | 69.75 | +| | C-Eval (val) | 31.83 | 41.4 | **59.01** | 58.8 | 37.47 | 40.36 | 50.13 | +| | AGI-Eval | 22.03 | 30.93 | 37.37 | **44.58** | 33.53 | 33.92 | 40.02 | +| Knowledge | BoolQ | 78.75 | 82.42 | 67 | **87.46** | 84.43 | 86.61 | 87.74 | +| | TriviaQA | 52.47 | 59.36 | 46.61 | 57.26 | **66.24** | 69.79 | 70.71 | +| | NaturalQuestions | 20.17 | 24.85 | 16.32 | 25.15 | **30.89** | 33.41 | 34.16 | +| Understanding | CMRC | 9.26 | 31.59 | 29.85 | **68.78** | 14.17 | 34.73 | 43.74 | +| | CSL | 55 | 58.75 | 63.12 | **65.62** | 57.5 | 59.38 | 60 | +| | RACE (middle) | 53.41 | 63.02 | 68.94 | **86.35** | 64.55 | 72.35 | 81.55 | +| | RACE (high) | 47.63 | 58.86 | 67.18 | **83.28** | 62.61 | 68.01 | 79.93 | +| | XSum | 20.37 | 23.37 | 25.23 | **35.54** | 20.55 | 19.91 | 25.38 | +| Reasoning | WinoGrande | 64.64 | 64.01 | 67.32 | **69.38** | 66.85 | 69.38 | 69.77 | +| | BBH | 37.93 | 45.62 | 48.98 | **52.51** | 49.98 | 58.38 | 64.91 | +| | GSM8K | 20.32 | 29.57 | **52.62** | **52.62** | 42.3 | 54.44 | 63.31 | +| | PIQA | 79.71 | 79.76 | 78.07 | 80.25 | **81.34** | 82.15 | 82.54 | +| Programming | HumanEval | 14.02 | 18.9 | 17.07 | **25.61** | 17.68 | 18.9 | 26.22 | +| | MBPP | 20.6 | 26.8 | 30.8 | **35.6** | 28.4 | 33.6 | 39.6 | + +Overall, InternLM-20B comprehensively outperforms open-source models in the 13B parameter range in terms of overall capabilities, and on inference evaluation sets, it approaches or even surpasses the performance of Llama-65B. + +- The evaluation results were obtained from [OpenCompass 20230920](https://github.com/internLM/OpenCompass/). +- The evaluation data may have numerical differences due to the version iteration of [OpenCompass](https://github.com/internLM/OpenCompass/), so please refer to the latest evaluation results of [OpenCompass](https://github.com/internLM/OpenCompass/). + +
+ + +
+ InternLM-7B + +#### News +[20230822] By utilizing richer SFT-type data, the InternLM-7B-Chat v1.1 model supports code interpretation and function invocation. The model structure and code remain unchanged, so the more powerful InternLM-7B-Chat v1.1 can be used in exactly the same way as InternLM-7B-Chat. + +#### Introduction +InternLM-7B contains a 7 billion parameter base model and a chat model tailored for practical scenarios. The model has the following characteristics: - It leverages trillions of high-quality tokens for training to establish a powerful knowledge base. - It supports an 8k context window length, enabling longer input sequences and stronger reasoning capabilities. - It provides a versatile toolset for users to flexibly build their own workflows. -Additionally, a lightweight training framework is offered to support model pre-training without the need for extensive dependencies. With a single codebase, it supports pre-training on large-scale clusters with thousands of GPUs, and fine-tuning on a single GPU while achieving remarkable performance optimizations. InternLM achieves nearly 90% acceleration efficiency during training on 1024 GPUs. - -## News - -InternLM-7B-Chat v1.1 is released with code interpreter and function calling capability. You can try it with [Lagent](https://github.com/InternLM/lagent). - -## InternLM-7B - -### Performance Evaluation +#### Performance Evaluation We conducted a comprehensive evaluation of InternLM using the open-source evaluation tool [OpenCompass](https://github.com/internLM/OpenCompass/). The evaluation covered five dimensions of capabilities: disciplinary competence, language competence, knowledge competence, inference competence, and comprehension competence. Here are some of the evaluation results, and you can visit the [OpenCompass leaderboard](https://opencompass.org.cn/rank) for more evaluation results. @@ -72,19 +149,12 @@ We conducted a comprehensive evaluation of InternLM using the open-source evalua - The evaluation results were obtained from [OpenCompass 20230706](https://github.com/internLM/OpenCompass/) (some data marked with *, which means come from the original papers), and evaluation configuration can be found in the configuration files provided by [OpenCompass](https://github.com/internLM/OpenCompass/). - The evaluation data may have numerical differences due to the version iteration of [OpenCompass](https://github.com/internLM/OpenCompass/), so please refer to the latest evaluation results of [OpenCompass](https://github.com/internLM/OpenCompass/). -### Model Zoo - -InternLM 7B and InternLM 7B Chat, trained using InternLM, have been open-sourced. We provide two formats of model weights for use. In addition to loading the models using the Transformers format, you can also load the weights directly using InternLM for further pre-training or human preference alignment training. - -| Model | InternLM Format Weight Download Link | Transformers Format Weight Download Link | -| ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- | -| **InternLM 7B** | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | [🤗internlm/intern-7b](https://huggingface.co/internlm/internlm-7b) | -| **InternLM Chat 7B v1.1** | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | [🤗internlm/intern-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | -| **InternLM Chat 7B** | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | [🤗internlm/intern-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) | -| **InternLM Chat 7B 8k** | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-8k) | [🤗internlm/intern-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) | +
**Limitations:** Although we have made efforts to ensure the safety of the model during the training process and to encourage the model to generate text that complies with ethical and legal requirements, the model may still produce unexpected outputs due to its size and probabilistic generation paradigm. For example, the generated responses may contain biases, discrimination, or other harmful content. Please do not propagate such content. We are not responsible for any consequences resulting from the dissemination of harmful information. +## Usage Examples + ### Import from Transformers To load the InternLM 7B Chat model using Transformers, use the following code: @@ -108,6 +178,23 @@ Sure, here are three tips for effective time management: Remember, good time management skills take practice and patience. Start with small steps and gradually incorporate these habits into your daily routine. ``` +### Import from ModelScope + +To load the InternLM model using ModelScope, use the following code: + +```python +from modelscope import snapshot_download, AutoTokenizer, AutoModelForCausalLM +import torch +model_dir = snapshot_download('Shanghai_AI_Laboratory/internlm-chat-7b-v1_1', revision='v1.0.0') +tokenizer = AutoTokenizer.from_pretrained(model_dir, device_map="auto", trust_remote_code=True,torch_dtype=torch.float16) +model = AutoModelForCausalLM.from_pretrained(model_dir,device_map="auto", trust_remote_code=True,torch_dtype=torch.float16) +model = model.eval() +response, history = model.chat(tokenizer, "hello", history=[]) +print(response) +response, history = model.chat(tokenizer, "please provide three suggestions about time management", history=history) +print(response) +``` + ### Dialogue You can interact with the InternLM Chat 7B model through a frontend interface by running the following code: @@ -124,45 +211,27 @@ The effect is as follows ### Deployment -We use [LMDeploy](https://github.com/InternLM/LMDeploy) to complete the workflow of InternLM deployment. +We use [LMDeploy](https://github.com/InternLM/LMDeploy) to complete the one-click deployment of InternLM. -```bash -python3 -m pip install lmdeploy +1. First, install LMDeploy: + +``` + python3 -m pip install lmdeploy ``` -You can utilize the following commands to conduct `internlm-chat-7b` FP16 inference, serve it and interact with AI assistant via WebUI: +2. Use the following command for quick deployment: -```bash -# convert weight layout -python3 -m lmdeploy.serve.turbomind.deploy internlm-chat-7b - -# inference lmdeploy's turbomind engine -python3 -m lmdeploy.turbomind.chat ./workspace - -# serving with gradio -python3 -m lmdeploy.serve.gradio.app ./workspace +``` + python3 -m lmdeploy.serve.turbomind.deploy InternLM-7B /path/to/internlm-7b/model hf ``` -You can also deploy 4-bit quantized `internlm-chat-7b` model via LMDeploy. It greatly trims down the model's memory overhead to 6G, just 40% of what FP16 inference would take. More importantly, with extreme optimized kernel, the inference performance achieves 2.4x faster than FP16 inference on A100-80G. +3. After exporting the model, you can start a server and have a conversation with the deployed model using the following command: -Try the followings to enjoy 4-bit `internlm-chat-7b` on a Geforce RTX 30x GPU card. You can find the inference benchmark from [here](https://github.com/InternLM/lmdeploy/blob/main/docs/en/w4a16.md#inference-performance). - -```bash -# download prequnantized internlm-chat-7b model from huggingface -git-lfs install -git clone https://huggingface.co/lmdeploy/llama2-chat-7b-w4 - -# Convert the model's layout and store it in the default path, ./workspace. -python3 -m lmdeploy.serve.turbomind.deploy internlm-chat-7b ./llama2-chat-7b-w4 awq --group-size 128 - -# inference lmdeploy's turbomind engine -python3 -m lmdeploy.turbomind.chat ./workspace - -# serving with gradio -python3 -m lmdeploy.serve.gradio.app ./workspace +``` + python3 -m lmdeploy.serve.client {server_ip_addresss}:33337 ``` -LMDeploy is an efficient toolkit for compressing, deploying, and serving LLM models. Please refer to the [deployment tutorial](https://github.com/InternLM/LMDeploy) for more details on deploying InternLM. +[LMDeploy](https://github.com/InternLM/LMDeploy) provides a complete workflow for deploying InternLM. Please refer to the [deployment tutorial](https://github.com/InternLM/LMDeploy) for more details on deploying InternLM. ## Fine-tuning & Training diff --git a/doc/imgs/modelscope_logo.png b/doc/imgs/modelscope_logo.png new file mode 100644 index 0000000000000000000000000000000000000000..0286d28f488f6425c9671d2b0ec81db56aede2f8 GIT binary patch literal 6186 zcmeHLXHyf}7CkfxC`DQ*(uJ!6A_!bXIwCeT9Dq6BBG%QLO=om zsfHpbMS71AAQS^qeYrFDAG~>=o*&LRGi%SB*|TSzwbwq+jc)6*F!M7509f>3*Wm!r z6$3!C4>>_aemkrfP!D=XO#@8;suE5f+A#pYtmmYyZR7>l)e_Ou(N>bXs-z?%e^Cwq z*z=q_zUE20JZjUs57|t$MmMU`T%Q6laSWu3-J*c~C9n%;=g?@6h-NnB;y#@XGiiG^ z#0<-cp36Unb<}(jeJ{r3{*`s%iINXN{^Yf_gOP(qVm9%x57o~WdYVHrajC!=T!lxd zD31y!7{c1RaiO%_uo-$0_9a5ABSSj?8+0hYU}UBGAx(t`paq^o7teBZ&i)Q-Fp@L_ zB8gx|jDvWR1`P*Y7w>j)fi7N9d8hgTGe`!&6&0u@2HvuO)t`E53}Cc0X#xsH3(k~6 zKpYLoIH!A-R^ubEHF*k`rp32{e2j+JMS6TWO{hh!ye@4hjsf_@o1@t@Y(Z#-mUIsg zxk3xHg*bcZMD*$MFAX-&l-hVkU|}=>Pe|!ayES;33!CzV-*;hwm|K32$@fCG6acN<8$${uf{@Gq|?6Tn@rY5>MecTvJOApv@ z_mjtL78i%03_4Nu@Wkt)-6?Y%%OUcc!l_40Nxm!q6KROQUCa1^;EE4k zr=g8uH6Qc=8o}E#vHT|!0lTAuJZcw!q~^T@4JjJXn^rUrK;sYI`)`t=`W+AeuIE3! zP^rmys)Y-KXU%RAY{Wxp?M__RICU1U!mf2ScGSO>xK{)M1m7`DF$B&BjL7*A&^kQ}}g_og4BFoWS z#$xSVPl{+uwVV^JVB%XcPq?ne-PhPhk9Ud4zc_r+s4JKJ^u00NH(GCv`ZvP-FFNwQ z29OGDR-vf{E(82ELaL#)1tkNwet086(2ICP=ztunL+p3E`>|0vw$IJ^B|CW#i{^@^ z35p0o^S^z#1@O6cWXM_Cm{87Dh;=x7tLrl~TGw3PT>Fj0s(1}Y1H_CeEL^VjFR`RI zhBXrY9D8kk_R3Xg^BWk+bx|SFTnX#HSUGKyU}ssoUoPBc7L`ED>z&kdfsMmtitZGt z4YE6B>r0Z*ZrxK0koTb&k+j4*G#lFREZqGXPniVxKn>5KE^ zwKDSG8uymweuKP?R?fSp=a%ndBzq^SlvBYwQ=&JEN1IS2Q#kY`)5+0k_mS)}vVmzb zOh_l~OWL5z55{HUa@bGbQ!J4zVVyAhb9^p*0emIAb$qfJ`mY(jre$>9y)+^(kl~ol zmhNFUMA{8sJAEYzB6HaXKfnv`i4oh(R?Y>H=cJx zrGh3OOO?v|;J)w#qkdzX!X}%eXZr}2_mL4d@*`n|cM)|Rd~u>Y9r6!1&9#P~n9`A? zUD6HgE#L30>iaMp2>8eb=-<)ydh393NOZ_qdfZFH+EF#LarJxl%|6jSmc9TPIvI7D zs$u6chqBWwXX;g zYMF`hnZmj}IiO2%`&meVu8@bx1Hyk_Qh+F-`CR3$AU1l?h#xNLPS}Bs@Yy$~36B~Q z+a47T=oS8jm0XWFJW5z>H$LSL3e^WxM3sHGOX!Ka7hg*2!+a8KV@_nL3slNMJ}cNZrMxo2r?jOev-# z7`@^9Bx#b{u$ilofLYe5Eafah?VGZ!vNoSAw`s09u7vZ0=P7sG$K+NISD9BcS5K^6 z-Lcz|*m2wG7wkF1Avk;uc`fh>(!0mU)w7XQG^_Bbw5{~hvYDfR<14S6ZAIwoTWA7D znok<^j$6r#68AiRLdqildc)v{uav^9k69CC-`&1V_NcZ8y*n_tm?h9{^+}bi`Ez5} zeC~12d`?MLN$WP6C~NIwZDDk$-6S@x+9V!49Gnr%c$j-|ZJ%(I@T++iwZ*e}o57Z*g=Uck_mux>HJuBcFntDn zF{2_w9K**GvZvIzwN5dySO`B!^1ws41ojY@Cmf|S2e>~&a=7ma>|g4U)E62)9Us{b znPw06c)h2;+*1pQp3!xU#tJ*}WkC0WxXxdYe|vGoxaJP>F>+UW7_(q;;E0RgOel@5 zzc%h%`^3i)1?KPAUV%6tvz388*Nj4KJP_Kbj`e+y^wh~ z6^STcoG7S=E-m*Fa(z`S6fMw!S0d6m(%!2thKXny$?_U5mbe=Ta&CoZMHuQd6~q-7 zSJh!}s=QpvF-_UIGI=dXd;d|S;SKMO%S?uFDO4ig%X;x`1JmonQ<(nmkH5=YP0miv zE`+l zF&IR6M0lTiWHe_EH`#JVO>D)(!v>3swWXe!DWsGpcN5&pf8MJ{aNDIkFd;jq4OyCo zT!5z7U_3Ez%70bfz%I?1-L@GDq@di!#-?u8eO@EMNJS)f%&tXq#Z8>E=FpeO4VE&2 z&A3GY8@HQN$XEW(l=$A7-p{>sigt=?mOZtY@vT=%g**N^Ayx#_qq28(P2VNQhMOmS z1GZVi1i7zGk&{;&KS=qRx!`nyDDH#n&n6T2?FAZ|mx3p;y1s_(xd{Qq0gO93WRf$| z!*!yfx%oYl8(b+)2&U}0m=)?VmbI4Y`@!?x1=d#0#g@OOUk3)v8~u3I_i5r&?{r2; z9R)&yp`-?to4qkIg>Lz(uW;lav4uH;)`GfCb-#nja`Oo9PQ?VX-29w#s+)TY&ywa# z9+c$f+*VipmN9|a)j!N|u`;&8IcoRkszZ_M`gJzD-P=>d1gnik&!1wu_Tv(Dx5`w@ zRds_sk7i*NmL8;syM#3}m2%`y1Jbnt^4E}kXZ(=Y$gh!$R(U8KxnotJ&Rufy3|69n z^2=vu4eHChu0LVrpWs(<*fe&u!nn`25>j(q^$_ni^UD@*+yBXejm*jvm3y50`)kzf zpl*-ukoXPpxP%;GVF4C?+}4A>q}h`9C7k`AgH3;kIH#q>9q zqe{1*kl&S7D0hl?3ODwlR5xEAmi3JXk+I~4=}w$*#<@d{W25bXeMD!;7sFjOV&L(i z>~8q>fsF2zeyYh7LBQZtv*|AhKxjAs`&20O2LRre09dgFKsg-%sAqPAu?jW7A z8~P`RjbYkbtl52;$T;mD=|PAZrtDT3;qme#)e!dm?oZw=xRB0i#`W(rrsh2(jELB3 zf5r(TUBb|6r4Y@=t9rH>^?Ek<=Lst`)J!I93b?3=#-5}4ViED1RH!SO>Y;)BANc>N zCe$FSda2)Rre;6KhRhh;+JS7G6Yl=jHV*Du=(W+E@o&?(pO2x6Mq7 z!R3(r>9f{stObV!(wFQ#h?P_LvlWq+>tSR2+w+rAvchg7*G64y(xvQ(N0`reG7YZFOGA!SX^c(IdFj z7jIRSUg6ux+ewKJSQ`4GT-S;b!Yjn=?c2tR-c-jFE;{>|Cxm`{AZJrSV$t6Tq+U8A zK2z>VkeGN<7f$|2<@1B<=czlNG4-EK<3=(q0O-@jf8akg`7a4kyF%j`7KBT$$Q$~H z#s9MTjS&992n8rv_KbW)#LiO%*L8$$<~2w0MzG-&)ssLe;2%+rqMX1YYCCxuH&2BA kZvP{@e?0rYbIdkdvI9y9J=8;;zW~tFxqZD{%l6rS0Ij2j9{>OV literal 0 HcmV?d00001 From 5e5d16068579725c958ab9cdba15154b40e79488 Mon Sep 17 00:00:00 2001 From: Shuo Zhang Date: Wed, 20 Sep 2023 16:26:43 +0800 Subject: [PATCH 02/11] fix(readme): fix readme about 20B releasing (#329) * fix(eval): StreamingDataset does not have an __len__ method. * doc(readme): update readme * update readme * update readme * update readme * update readme * update readme --- README-zh-Hans.md | 22 +++++++++++----------- README.md | 24 ++++++++++-------------- 2 files changed, 21 insertions(+), 25 deletions(-) diff --git a/README-zh-Hans.md b/README-zh-Hans.md index 4ab9a84..288a6e2 100644 --- a/README-zh-Hans.md +++ b/README-zh-Hans.md @@ -40,7 +40,7 @@ InternLM 是一个开源的轻量级训练框架，旨在支持大模型训练而无需大量的依赖。通过单一的代码库，它支持在拥有数千个 GPU 的大型集群上进行预训练，并在单个 GPU 上进行微调，同时实现了卓越的性能优化。在1024个 GPU 上训练时，InternLM 可以实现近90%的加速效率。 -基于InternLM训练框架，我们已经预训练了两个开源的预训练模型：InternLM-7B 和 InternLM-20B。 +基于InternLM训练框架，我们已经发布了两个开源的预训练模型：InternLM-7B 和 InternLM-20B。 ## 更新 @@ -52,20 +52,20 @@ InternLM 是一个开源的轻量级训练框架，旨在支持大模型训练 我们的模型在三个平台上发布：Transformers、ModelScope 和 OpenXLab。 -| Model | Transformers | ModelScope | OpenXLab | | 发布日期 | -|---------------------------|------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|---|--------------| -| **InternLM Chat 20B** | [🤗internlm/internlm-chat-20b](https://huggingface.co/internlm/internlm-20b-chat) | [ Shanghai_AI_Laboratory/internlm-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b-chat/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-20b) | | 2023-09-20 | -| **InternLM 20B** | [🤗internlm/internlm-20b](https://huggingface.co/internlm/internlm-20b) | [ Shanghai_AI_Laboratory/internlm-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-20b) | | 2023-09-20 | -| **InternLM Chat 7B v1.1** | [🤗internlm/internlm-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | [ Shanghai_AI_Laboratory/internlm-chat-7b-v1_1](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-v1_1/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | | 2023-08-22 | -| **InternLM 7B** | [🤗internlm/internlm-7b](https://huggingface.co/internlm/internlm-7b) | [ Shanghai_AI_Laboratory/internlm-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | | 2023-07-06 | -| **InternLM Chat 7B** | [🤗internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) | [ Shanghai_AI_Laboratory/internlm-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | | 2023-07-06 | -| **InternLM Chat 7B 8k** | [🤗internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) | [ Shanghai_AI_Laboratory/internlm-chat-7b-8k](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-8k/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-8k) | | 2023-07-06 | +| Model | Transformers | ModelScope | OpenXLab |发布日期 | +|---------------------------|------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| +| **InternLM Chat 20B** | [🤗internlm/internlm-chat-20b](https://huggingface.co/internlm/internlm-20b-chat) | [ Shanghai_AI_Laboratory/internlm-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b-chat/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-20b) | 2023-09-20 | +| **InternLM 20B** | [🤗internlm/internlm-20b](https://huggingface.co/internlm/internlm-20b) | [ Shanghai_AI_Laboratory/internlm-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-20b) | 2023-09-20 | +| **InternLM Chat 7B v1.1** | [🤗internlm/internlm-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | [ Shanghai_AI_Laboratory/internlm-chat-7b-v1_1](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-v1_1/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | 2023-08-22 | +| **InternLM 7B** | [🤗internlm/internlm-7b](https://huggingface.co/internlm/internlm-7b) | [ Shanghai_AI_Laboratory/internlm-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | 2023-07-06 | +| **InternLM Chat 7B** | [🤗internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) | [ Shanghai_AI_Laboratory/internlm-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | 2023-07-06 | +| **InternLM Chat 7B 8k** | [🤗internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) | [ Shanghai_AI_Laboratory/internlm-chat-7b-8k](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-8k/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-8k) | 2023-07-06 |
InternLM-20B -### 简介 +#### 简介 InternLM-20B 在超过 **2.3T** Tokens 包含高质量英文、中文和代码的数据上进行预训练，其中 Chat 版本还经过了 SFT 和 RLHF 训练，使其能够更好、更安全地满足用户的需求。 InternLM 20B 在模型结构上选择了深结构，InternLM-20B 的层数设定为60层，超过常规7B和13B模型所使用的32层或者40层。在参数受限的情况下，提高层数有利于提高模型的综合能力。此外，相较于InternLM-7B，InternLM-20B使用的预训练数据经过了更高质量的清洗，并补充了高知识密度和用于强化理解和推理能力的训练数据。因此，它在理解能力、推理能力、数学能力、编程能力等考验语言模型技术水平的方面都得到了显著提升。总体而言，InternLM-20B具有以下的特点： @@ -74,7 +74,7 @@ InternLM 20B 在模型结构上选择了深结构，InternLM-20B 的层数设定 - 支持16k语境长度（通过推理时外推） - 更好的价值对齐 -### 性能对比 +#### 性能对比 在OpenCompass提出的5个能力维度上，InternLM-20B都取得很好的效果（粗体为13B-33B这个量级范围内，各项最佳成绩） diff --git a/README.md b/README.md index f4b9351..48016ab 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ ## Introduction InternLM is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies. With a single codebase, it supports pre-training on large-scale clusters with thousands of GPUs, and fine-tuning on a single GPU while achieving remarkable performance optimizations. InternLM achieves nearly 90% acceleration efficiency during training on 1024 GPUs. -Based on the InternLM training framework, we have pre-trained two open-sourced pretrained model InternLM-7B and InternLM-20B. +Based on the InternLM training framework, we have released two open-sourced pretrained model InternLM-7B and InternLM-20B. ## News @@ -52,19 +52,15 @@ Based on the InternLM training framework, we have pre-trained two open-sourced p Our models are released in three platforms: Transformers, ModelScope and OpenXLab. -| Model | Transformers | ModelScope | OpenXLab | | Release Date | -|---------------------------|------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|---|--------------| -| **InternLM Chat 20B** | [🤗internlm/internlm-chat-20b](https://huggingface.co/internlm/internlm-20b-chat) | [ Shanghai_AI_Laboratory/internlm-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b-chat/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-20b) | | 2023-09-20 | -| **InternLM 20B** | [🤗internlm/internlm-20b](https://huggingface.co/internlm/internlm-20b) | [ Shanghai_AI_Laboratory/internlm-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-20b) | | 2023-09-20 | -| **InternLM Chat 7B v1.1** | [🤗internlm/internlm-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | [ Shanghai_AI_Laboratory/internlm-chat-7b-v1_1](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-v1_1/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | | 2023-08-22 | -| **InternLM 7B** | [🤗internlm/internlm-7b](https://huggingface.co/internlm/internlm-7b) | [ Shanghai_AI_Laboratory/internlm-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | | 2023-07-06 | -| **InternLM Chat 7B** | [🤗internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) | [ Shanghai_AI_Laboratory/internlm-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | | 2023-07-06 | -| **InternLM Chat 7B 8k** | [🤗internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) | [ Shanghai_AI_Laboratory/internlm-chat-7b-8k](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-8k/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-8k) | | 2023-07-06 | +| Model | Transformers | ModelScope | OpenXLab | Release Date | +|---------------------------|------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| +| **InternLM Chat 20B** | [🤗internlm/internlm-chat-20b](https://huggingface.co/internlm/internlm-20b-chat) | [ Shanghai_AI_Laboratory/internlm-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b-chat/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-20b) | 2023-09-20 | +| **InternLM Chat 7B v1.1** | [🤗internlm/internlm-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | [ Shanghai_AI_Laboratory/internlm-chat-7b-v1_1](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-v1_1/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | 2023-08-22 | +| **InternLM 7B** | [🤗internlm/internlm-7b](https://huggingface.co/internlm/internlm-7b) | [ Shanghai_AI_Laboratory/internlm-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | 2023-07-06 | +| **InternLM Chat 7B** | [🤗internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) | [ Shanghai_AI_Laboratory/internlm-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | 2023-07-06 | +| **InternLM Chat 7B 8k** | [🤗internlm/internlm-chat-7b-8k](https://huggingface.co/internlm/internlm-chat-7b-8k) | [ Shanghai_AI_Laboratory/internlm-chat-7b-8k](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-8k/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-8k) | 2023-07-06 | -
- InternLM-20B - -### Introduction +#### Introduction InternLM-20B was pre-trained on over **2.3T** Tokens containing high-quality English, Chinese, and code data. Additionally, the Chat version has undergone SFT and RLHF training, enabling it to better and more securely meet users' needs. In terms of model structure, InternLM-20B opted for a deeper architecture, with a depth set at 60 layers. This surpasses the conventional 7B and 13B models that utilize 32 or 40 layers. When parameters are limited, increasing the number of layers can enhance the model's overall capability. Furthermore, compared to InternLM-7B, the pre-training data used for InternLM-20B underwent higher quality cleansing and was supplemented with data rich in knowledge and designed for reinforcing understanding and reasoning capabilities. As a result, it exhibits significant improvements in understanding, reasoning, mathematical, and programming abilities—all of which test the technical proficiency of language models. Overall, InternLM-20B features the following characteristics: @@ -73,7 +69,7 @@ In terms of model structure, InternLM-20B opted for a deeper architecture, with - Supports a 16k context length (Through inference extrapolation) - Better value alignment. -### Performace Evaluation +#### Performance Evaluation On the 5 capability dimensions proposed by OpenCompass, InternLM-20B has achieved excellent results (the bolded scores represent the best performances within the 13B-33B parameter range). From e6118174422224334bb14c5523caea6391fcaaf8 Mon Sep 17 00:00:00 2001 From: Shuo Zhang Date: Wed, 20 Sep 2023 16:46:45 +0800 Subject: [PATCH 03/11] fix(doc): add 20b releasing info to readme (#330) * fix(eval): StreamingDataset does not have an __len__ method. * doc(readme): update readme * update readme * update readme * update readme * update readme * update readme * update readme --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 48016ab..59b1f63 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,7 @@ Our models are released in three platforms: Transformers, ModelScope and OpenXLa | Model | Transformers | ModelScope | OpenXLab | Release Date | |---------------------------|------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------| | **InternLM Chat 20B** | [🤗internlm/internlm-chat-20b](https://huggingface.co/internlm/internlm-20b-chat) | [ Shanghai_AI_Laboratory/internlm-chat-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b-chat/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-20b) | 2023-09-20 | +| **InternLM 20B** | [🤗internlm/internlm-20b](https://huggingface.co/internlm/internlm-20b) | [ Shanghai_AI_Laboratory/internlm-20b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-20b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-20b) | 2023-09-20 | | **InternLM Chat 7B v1.1** | [🤗internlm/internlm-chat-7b-v1.1](https://huggingface.co/internlm/internlm-chat-7b-v1.1) | [ Shanghai_AI_Laboratory/internlm-chat-7b-v1_1](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b-v1_1/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b-v1.1) | 2023-08-22 | | **InternLM 7B** | [🤗internlm/internlm-7b](https://huggingface.co/internlm/internlm-7b) | [ Shanghai_AI_Laboratory/internlm-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-7b) | 2023-07-06 | | **InternLM Chat 7B** | [🤗internlm/internlm-chat-7b](https://huggingface.co/internlm/internlm-chat-7b) | [ Shanghai_AI_Laboratory/internlm-chat-7b](https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-chat-7b/summary) | [![Open in OpenXLab](https://cdn-static.openxlab.org.cn/header/openxlab_models.svg)](https://openxlab.org.cn/models/detail/OpenLMLab/InternLM-chat-7b) | 2023-07-06 | From d1e52f0c036454a82f9f50b5ab45986a547a281b Mon Sep 17 00:00:00 2001 From: Guoteng <32697156+SolenoidWGT@users.noreply.github.com> Date: Fri, 22 Sep 2023 18:45:33 +0800 Subject: [PATCH 04/11] feat(doc/code-docs): add checkpoint save/load usage doc (#311) * feat(doc): add checkpoint doc * fix checkpoint doc * fix comment * fix(doc/code-docs): remove fuzzy * fix(doc/code-docs): fix some errors * fix(doc/code-docs): minor fix --------- Co-authored-by: li126com Co-authored-by: huangting4201 <1538303371@qq.com> --- .../locales/en/LC_MESSAGES/checkpoint.po | 339 +++++++++++++++++- doc/code-docs/source/checkpoint.rst | 166 ++++++++- doc/imgs/ckpt_path_format_CN.png | Bin 0 -> 156419 bytes 3 files changed, 485 insertions(+), 20 deletions(-) create mode 100644 doc/imgs/ckpt_path_format_CN.png diff --git a/doc/code-docs/locales/en/LC_MESSAGES/checkpoint.po b/doc/code-docs/locales/en/LC_MESSAGES/checkpoint.po index bd81fa5..e82a9b1 100644 --- a/doc/code-docs/locales/en/LC_MESSAGES/checkpoint.po +++ b/doc/code-docs/locales/en/LC_MESSAGES/checkpoint.po @@ -3,12 +3,11 @@ # This file is distributed under the same license as the InternLM package. # FIRST AUTHOR , 2023. # -#, fuzzy msgid "" msgstr "" "Project-Id-Version: InternLM \n" "Report-Msgid-Bugs-To: \n" -"POT-Creation-Date: 2023-09-13 17:07+0800\n" +"POT-Creation-Date: 2023-09-15 19:06+0800\n" "PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n" "Last-Translator: FULL NAME \n" "Language: en\n" @@ -20,7 +19,7 @@ msgstr "" "Generated-By: Babel 2.12.1\n" #: ../../source/checkpoint.rst:2 -msgid "模型保存" +msgid "模型加载与保存" msgstr "Model Checkpointing" #: ../../source/checkpoint.rst:4 @@ -36,12 +35,86 @@ msgstr "" #: ../../source/checkpoint.rst:6 msgid "InternLM支持启动时自动加载最新的模型备份，并在接收信号退出训练时自动进行模型备份。" -msgstr "InternLM supports automatic loading of latest ckpt at startup and automatic model checkpointing at signal quit. " +msgstr "InternLM supports automatic loading of latest ckpt at startup and automatic model checkpointing at signal quit." #: ../../source/checkpoint.rst:9 -msgid "Checkpointing" +msgid "CheckpointManager" msgstr "" +#: ../../source/checkpoint.rst:11 +msgid "" +"``CheckpointManager`` " +"是InternLM负责进行模型加载和保存的工具类，其会使用config文件中的ckpt字段的初始化参数字典初始化自身的参数，目前相关的参数有：" +msgstr "" +"CheckpointManager is the utility class within InternLM responsible for " +"model loading and saving. It initializes its own parameters using the " +"initialization parameter dictionary from the 'ckpt' field in the config " +"file. Currently, the relevant parameters are as follows" + +#: ../../source/checkpoint.rst:13 +msgid "``enable_save_ckpt``: 是否开启检查点存储功能（不影响检查点加载）。参数类型 ``bool``，必选参数。" +msgstr "" +"``enable_save_ckpt``: Whether to enable checkpoint storage functionality " +"(does not affect checkpoint loading). Parameter type: `bool`, it is a " +"required parameter." + +#: ../../source/checkpoint.rst:15 +msgid "``save_ckpt_folder``: 检查点存储路径，参数类型 ``str``，默认为： ``None``，在开启检查点存储功能时为必选参数。" +msgstr "" +"``save_ckpt_folder``: Checkpoint storage path. Parameter type: ``str``. " +"This is a required parameter when enabling checkpoint storage " +"functionality." + +#: ../../source/checkpoint.rst:17 +msgid "``checkpoint_every``: 检查点存储频率，参数类型 ``int``，默认为： ``50``。" +msgstr "" +"``checkpoint_every``: Checkpoint storage frequency. Parameter type: " +"``int``." + +#: ../../source/checkpoint.rst:19 +msgid "" +"``load_ckpt_folder``: 初始化检查点/权重加载路径。参数类型 ``str``，默认为： ``None``，详见 :ref" +":`load-ckpt-folder`。" +msgstr "" +"``load_ckpt_folder``: Initialization checkpoint/weight loading path. " +"Parameter type: ``str``. Default is ``None``. :ref:`load-ckpt-folder`" + +#: ../../source/checkpoint.rst:21 +msgid "``async_upload``: 是否开启异步上传，默认值为：``False``，详见 :ref:`asyncupload`。" +msgstr "" +"``async_upload``: Whether to enable asynchronous uploading. See " +"documentation for more details :ref:`asyncupload`" + +#: ../../source/checkpoint.rst:23 +msgid "``async_upload_tmp_folder``: 异步上传临时存储路径。" +msgstr "" +"``async_upload_tmp_folder``: Temporary storage path for asynchronous " +"uploading." + +#: ../../source/checkpoint.rst:25 +msgid "" +"``oss_snapshot_freq``: 快照存储频率，默认值为：``checkpoint_every``的一半。详见 " +":ref:`snapshot`。" +msgstr "" +"``oss_snapshot_freq``: Snapshot storage frequency. See documentation for " +"more details :ref:`snapshot`." + +#: ../../source/checkpoint.rst:27 +msgid "``auto_resume``: 是否开启检查点自动恢复，默认值为：``True``，详见 :ref:`autoresume`。" +msgstr "" +"``auto_resume``: Whether to enable automatic checkpoint resume. See " +"documentation for more details :ref:`autoresume`." + +#: ../../source/checkpoint.rst:29 +msgid "``stop_file_path`` : 检查点存储控制文件的路径，默认值为：``None``，详见 :ref:`stopfile`。" +msgstr "" +"``stop_file_path``: Path to the checkpoint storage control file. See " +"documentation for more details :ref:`stopfile`." + +#: ../../source/checkpoint.rst:32 +msgid "下面给出config文件的参数设置例子：" +msgstr "Here is an example of parameter settings in the config file." + #: internlm.utils.model_checkpoint.CheckpointManager:1 of msgid "StorageManagerContext" msgstr "" @@ -86,21 +159,253 @@ msgstr "" msgid "Save checkpoint to the given folder path." msgstr "" -#~ msgid "Attempt to restore the training state of the last ckpt." -#~ msgstr "" +#: ../../source/checkpoint.rst:53 +msgid "加载与存储格式约定" +msgstr "Model loading and saving path format conventions." -#~ msgid "lr_scheduler object." -#~ msgstr "" +#: ../../source/checkpoint.rst:58 +msgid "(1) 路径格式约定" +msgstr "(1) Path format conventions." -#~ msgid "optimizer object." -#~ msgstr "" +#: ../../source/checkpoint.rst:60 +msgid "InternLM对config中出现的所有存储路径都遵循以下的路径格式约定:" +msgstr "" +"InternLM follows the following path format conventions for all storage " +"paths specified in the config:" -#~ msgid "learning rate." -#~ msgstr "" +#: ../../source/checkpoint.rst:66 +msgid "对于不同backend的路径，有以下的规则需要注意:" +msgstr "For paths of different backends, the following rules should be noted:" -#~ msgid "traing states." -#~ msgstr "" +#: ../../source/checkpoint.rst:68 +msgid "" +"如果需要使用boto3的路径，需要在运行前提前导入 ``S3_ACCESS_KEY_ID`` 和 " +"``S3_SECRET_ACCESS_KEY_ID`` 这两个环境变量。" +msgstr "" +"If you need to use paths with Boto3, make sure to import the " +"``S3_ACCESS_KEY_ID`` and ``S3_SECRET_ACCESS_KEY_ID`` environment " +"variables before running." -#~ msgid "traning dataloader object" -#~ msgstr "" +#: ../../source/checkpoint.rst:70 +msgid "bucket的endpoint一般分为Inside IP和Outside IP，如果可以尽量使用inside IP，会获得更佳的存储速度。" +msgstr "" +"The bucket's endpoint is typically divided into Inside IP and Outside IP." +" Whenever possible, it's advisable to use the Inside IP to achieve better" +" storage speed." +#: ../../source/checkpoint.rst:75 +msgid "(2) 模型加载(load_ckpt_folder)格式约定" +msgstr "(2) Model loading format conventions (load_ckpt_folder)." + +#: ../../source/checkpoint.rst:77 +msgid "load_ckpt_folder 由三个字段组成， ``path`` 、 ``content`` 和 ``ckpt_type`` 。" +msgstr "" +"``load_ckpt_folder`` consists of three fields: ``path``, ``content``, and" +" ``ckpt_type``." + +#: ../../source/checkpoint.rst:79 +msgid "``path``：给出了检查点/初始化模型权重的加载路径（path的格式见下小节）" +msgstr "" +"``path``: Specifies the loading path for the checkpoint/initial model " +"weights (the format of the path is described in the following " +"subsection)." + +#: ../../source/checkpoint.rst:81 +msgid "``content``: 表示需要加载的内容，目前支持的字段包括：" +msgstr "" +"``content``: Indicates the content to be loaded, currently supported " +"fields include:" + +#: ../../source/checkpoint.rst:83 +msgid "``model``：加载模型权重。" +msgstr "``model``: Load model weights." + +#: ../../source/checkpoint.rst:84 +msgid "``sampler``：加载sampler状态。" +msgstr "``sampler``: Load sampler state." + +#: ../../source/checkpoint.rst:85 +msgid "``scheduler``：加载lr_scheduler状态。" +msgstr "``scheduler``: Load lr_scheduler state." + +#: ../../source/checkpoint.rst:86 +msgid "``optimzier``：加载optimizer状态。" +msgstr "``optimizer``: Load optimizer state." + +#: ../../source/checkpoint.rst:87 +msgid "``all``：表示所有状态均加载，一般在resume训练使用。" +msgstr "" +"``all``: Indicates that all states should be loaded, typically used for " +"resuming training." + +#: ../../source/checkpoint.rst:89 +msgid "``ckpt_type``：表示加载的模型权重类型，目前支持的字段包括：" +msgstr "" +"``ckpt_type``: Represents the type of model weight to be loaded, " +"currently supported fields include:" + +#: ../../source/checkpoint.rst:91 +msgid "``internlm``：internlm约定的checkpoint存储格式。" +msgstr "``internlm``: Checkpoint storage format as per InternLM conventions." + +#: ../../source/checkpoint.rst:93 +msgid "下面给出两个例子：" +msgstr "Here are two examples:" + +#: ../../source/checkpoint.rst:107 +msgid "异步上传" +msgstr "Asynchronous upload." + +#: ../../source/checkpoint.rst:109 +msgid "" +"异步上传会先同步的将模型存储到 ``async_upload_tmp_folder`` " +"中，再异步的写入远端存储（OSS/NFS）中。从而避免存储ckpt阻塞训练过长时间。" +msgstr "" +"Asynchronous upload first synchronously stores the model in the " +"``async_upload_tmp_folder`` and then asynchronously writes it to remote " +"storage (OSS/NFS). This helps prevent blocking training for extended " +"periods while storing checkpoints." + +#: ../../source/checkpoint.rst:111 ../../source/checkpoint.rst:129 +#: ../../source/checkpoint.rst:145 ../../source/checkpoint.rst:160 +msgid "config.ckpt 中相关的参数：" +msgstr "The parameters related to ``config.ckpt`` are:" + +#: ../../source/checkpoint.rst:113 +msgid "``async_upload``: 是否开启异步上传。参数类型 ``bool/None``，默认为 ``False``。" +msgstr "" +"``async_upload``: Whether to enable asynchronous upload. Parameter type: " +"``bool/None``. Default is ``False``." + +#: ../../source/checkpoint.rst:115 +msgid "" +"``async_upload_tmp_folder``: 异步上传临时存储路径。参数类型 ``str/None``, 默认值为 " +"``/dev/shm/{JOB_NAME}_tmp_ckpt/``。" +msgstr "" +"`async_upload_tmp_folder`: Temporary storage path for asynchronous " +"upload. Parameter type: `str/None`. Default value is " +"``/dev/shm/{JOB_NAME}_tmp_ckpt/``." + +#: ../../source/checkpoint.rst:117 +msgid "需要注意的是，异步上传功能仅在backend为boto3时才会有效果，bcakend为local时只支持同步存储。" +msgstr "" +"It's important to note that asynchronous upload functionality is only " +"effective when the backend is set to \"boto3.\" When the backend is set " +"to \"local,\" only synchronous storage is supported." + +#: ../../source/checkpoint.rst:119 +msgid "" +"``async_upload_tmp_folder`` " +"设置的的原则为尽量设置为计算节点的local目录，这样才可以获得最佳的异步上传速度，一般来说建议为 ``/dev/shm`` 或 " +"``/nvme`` 下的路径，如果使用同步上传，则该路径可不给。" +msgstr "" +"The setting principle is to try to set it to the local directory of the " +"computing node, so as to obtain the best asynchronous upload speed. " +"Generally speaking, it is recommended to use the path under ``/dev/shm`` " +"or ``/nvme``. If If you use synchronous upload, this path does not need " +"to be given." + +#: ../../source/checkpoint.rst:125 +msgid "快照检查点" +msgstr "Snapshot Checkpoint" + +#: ../../source/checkpoint.rst:127 +msgid "" +"快照检查点是一种特殊的检查点，其是为了减少模型因为训练崩溃（ECC error, NCCL error, " +".etc）等问题导致训练任务崩溃而损失的训练进度。其采用交替覆盖写的策略，所占用的存储大小为两个step的检查点所需的空间。配合上异步的检查点写入，在不影响训练速度和存储容量的条件下极大的增大了检查点的存储频率。" +msgstr "" +"Snapshot checkpoint is a special checkpoint that is used to reduce the " +"loss of training progress due to training task crashes caused by problems" +" such as training crashes (ECC error, NCCL error.etc). It adopts an " +"alternating overwriting strategy, and the storage size occupied is the " +"space required for the checkpoints of two steps. Coupled with " +"asynchronous checkpoint writing, it greatly increases the storage " +"frequency of checkpoints without affecting training speed and storage " +"capacity." + +#: ../../source/checkpoint.rst:131 +msgid "``oss_snapshot_freq``: 快照存储频率。参数类型 ``int/None``，默认为 ``50``。" +msgstr "" +"``oss_snapshot_freq``: Snapshot storage frequency. Parameter type " +"``int/None``, default is ``50``" + +#: ../../source/checkpoint.rst:133 +msgid "" +"``oss_snapshot_freq`` 可以根据模型每step时间酌情设置，一般快照频率在1小时以下，半小时以上为怡/不给（默认值是 " +"``checkpoint_every`` 的二分之一）。" +msgstr "" +"``oss_snapshot_freq`` can be set according to the time of each step of " +"the model. Generally, the snapshot frequency is less than 1 hour, and it " +"is Yi/Non for more than half an hour (the default value is one-half of " +"``checkpoint_every``)" + +#: ../../source/checkpoint.rst:139 +msgid "检查点自动恢复" +msgstr "Checkpoint automatic recovery" + +#: ../../source/checkpoint.rst:141 +msgid "" +"检查点自动加载功能的目的是在resume训练时，自动加载 ``save_ckpt_folder`` " +"路径下最新的检查点（包括snapshot检查点）。配合上自动重启机制，可以实现无人干预的任务自动恢复。" +msgstr "" +"The purpose of Checkpoint automatic recovery is to automatically load the" +" latest checkpoint (including snapshot checkpoint) under the " +"``save_ckpt_folder`` path during resume training. Coupled with the " +"automatic restart mechanism, tasks can be automatically restored without " +"human intervention." + +#: ../../source/checkpoint.rst:143 +msgid "" +"该功能默认开启，所以要注意如果需要加载 ``load_ckpt_folder`` 路径下的模型权重，要将 ``auto_resume`` 设置为 " +"False，否则可能会产生预期外的行为。" +msgstr "" +"This function is enabled by default, so please note that if you need to " +"load the model weights under the ``load_ckpt_folder`` path, you must set " +"``auto_resume`` to ``False``, otherwise unexpected behavior may occur." + +#: ../../source/checkpoint.rst:147 +msgid "``auto_resume``: 是否开启检查点自动恢复。参数类型 ``bool``，默认为 ``True``。" +msgstr "" +"``auto_resume``: Whether to enable automatic checkpoint recovery. " +"Parameter type ``bool``, default is ``True``" + +#: ../../source/checkpoint.rst:149 +msgid "" +"``auto_resume`` 如果为True，则尝试从 ``save_ckpt_folder`` " +"路径中自动加载最新的ckpt，如果找不到，则从step 0开始训练。如果为False，则尝试从 ``load_ckpt_folder`` " +"中加载模型参数。" +msgstr "" +"``auto_resume`` If True, attempts to save_ckpt_folder`Automatically load " +"the latest ckpt in the path. If not found, training will start from step " +"0. If False, try to load model parameters from ``load_ckpt_folder``" + +#: ../../source/checkpoint.rst:155 +msgid "手动控制检查点存储" +msgstr "Manual control of checkpoint storage" + +#: ../../source/checkpoint.rst:157 +msgid "" +"在模型距离下一次检查点存储还有很长时间，这时如果希望立刻停止一个任务，又不希望丢失目前训练进度时可以使用手动控制检查点存储功能。通过向一个位于NFS上的" +" ``stop_file_path`` 文件中写入希望任务停止的step步数，Global Rank " +"0的进程会在每个step轮询该文件的值，如果发现有我们给出的停止step，则会进行一次广播通知所有的训练进程，约定各进程在训练到该step时存储一个检查点，并选择是否退出。" +msgstr "" +"When the model is still a long time away from the next checkpoint " +"storage, if you want to stop a task immediately and do not want to lose " +"the current training progress, you can use the manual control checkpoint " +"storage function. By writing the number of steps you want the task to " +"stop to a ``stop_file_path`` file located on NFS, the Global Rank 0 " +"process will poll the value of the file at each step. If it finds that " +"there is a stop step we gave , a broadcast will be performed to notify " +"all training processes, and it is agreed that each process will store a " +"checkpoint when training reaches this step, and choose whether to exit." + +#: ../../source/checkpoint.rst:162 +msgid "``stop_file_path``：检查点存储控制文件的路径，参数类型 ``str/None``，默认为 ``None``，表示关闭该功能。" +msgstr "" +"``stop_file_path``: The path of the checkpoint storage control file, " +"parameter type ``str/None``, the default is ``None``, indicating to turn " +"off this function" + +#: ../../source/checkpoint.rst:164 +msgid "下面给出一个写入 ``stop_file_path`` 的例子：" +msgstr "An example of writing to ``stop_file_path`` is given below:" diff --git a/doc/code-docs/source/checkpoint.rst b/doc/code-docs/source/checkpoint.rst index ee4f037..cd9b755 100644 --- a/doc/code-docs/source/checkpoint.rst +++ b/doc/code-docs/source/checkpoint.rst @@ -1,12 +1,172 @@ -模型保存 +模型加载与保存 =================== InternLM 使用 ``internlm.utils.model_checkpoint.CheckpointManager`` 来管理模型保存。其中，可以使用 ``CheckpointManager.try_save_checkpoint(train_state)`` 来保存指定 step 的模型状态。 InternLM支持启动时自动加载最新的模型备份，并在接收信号退出训练时自动进行模型备份。 -Checkpointing -------------- +CheckpointManager +-------------------------- + +``CheckpointManager`` 是InternLM负责进行模型加载和保存的工具类，其会使用config文件中的ckpt字段的初始化参数字典初始化自身的参数，目前相关的参数有： + +- ``enable_save_ckpt``: 是否开启检查点存储功能（不影响检查点加载）。参数类型 ``bool``，必选参数。 + +- ``save_ckpt_folder``: 检查点存储路径，参数类型 ``str``，默认为： ``None``，在开启检查点存储功能时为必选参数。 + +- ``checkpoint_every``: 检查点存储频率，参数类型 ``int``，默认为： ``50``。 + +- ``load_ckpt_folder``: 初始化检查点/权重加载路径。参数类型 ``str``，默认为： ``None``，详见 :ref:`load-ckpt-folder`。 + +- ``async_upload``: 是否开启异步上传，默认值为：``False``，详见 :ref:`asyncupload`。 + +- ``async_upload_tmp_folder``: 异步上传临时存储路径。 + +- ``oss_snapshot_freq``: 快照存储频率，默认值为：``checkpoint_every``的一半。详见 :ref:`snapshot`。 + +- ``auto_resume``: 是否开启检查点自动恢复，默认值为：``True``，详见 :ref:`autoresume`。 + +- ``stop_file_path`` : 检查点存储控制文件的路径，默认值为：``None``，详见 :ref:`stopfile`。 + + +下面给出config文件的参数设置例子： + +.. code-block:: python + + ckpt = dict( + enable_save_ckpt=False, # enable ckpt save. + save_ckpt_folder=SAVE_CKPT_FOLDER, # Path to save training ckpt. + load_ckpt_folder=dict(path="local:/mnt/mfs/ckpt", content=["all",], ckpt_type="internlm"), + auto_resume=False, # disable auto-resume, internlm will load model checkpoint from the path of 'load_ckpt_folder'. + checkpoint_every=CHECKPOINT_EVERY, + async_upload=True, # async ckpt upload. (only work for boto3 ckpt) + async_upload_tmp_folder="/dev/shm/internlm_tmp_ckpt/", # path for temporarily files during asynchronous upload. + oss_snapshot_freq=int(CHECKPOINT_EVERY / 2), # snapshot ckpt save frequency. + ) + .. autoclass:: internlm.utils.model_checkpoint.CheckpointManager :members: + + +加载与存储格式约定 +-------------------------- + +.. _load-ckpt-folder: + +(1) 路径格式约定 +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +InternLM对config中出现的所有存储路径都遵循以下的路径格式约定: + +.. figure:: ../../imgs/ckpt_path_format_CN.png + :scale: 30% + :class: with-border + +对于不同backend的路径，有以下的规则需要注意: + +1. 如果需要使用boto3的路径，需要在运行前提前导入 ``S3_ACCESS_KEY_ID`` 和 ``S3_SECRET_ACCESS_KEY_ID`` 这两个环境变量。 + +2. bucket的endpoint一般分为Inside IP和Outside IP，如果可以尽量使用inside IP，会获得更佳的存储速度。 + + + +(2) 模型加载(load_ckpt_folder)格式约定 +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +load_ckpt_folder 由三个字段组成， ``path`` 、 ``content`` 和 ``ckpt_type`` 。 + +- ``path``：给出了检查点/初始化模型权重的加载路径（path的格式见下小节） + +- ``content``: 表示需要加载的内容，目前支持的字段包括： + + - ``model``：加载模型权重。 + - ``sampler``：加载sampler状态。 + - ``scheduler``：加载lr_scheduler状态。 + - ``optimzier``：加载optimizer状态。 + - ``all``：表示所有状态均加载，一般在resume训练使用。 + +- ``ckpt_type``：表示加载的模型权重类型，目前支持的字段包括： + + - ``internlm``：internlm约定的checkpoint存储格式。 + +下面给出两个例子： + +.. code-block:: python + + # 从文件存储相对路径 ckpt_model 中加载已有模型权重初始化模型，适合 sft 等训练初始化 + load_ckpt_folder= dict(path="local:ckpt_model", content=["model",], ckpt_type="internlm") + + # 从文件存储相对路径 ckpt_model 中加载所有的状态，适合断点续训的场景 + load_ckpt_folder= dict(path="local:ckpt_model", content=["all",], ckpt_type="internlm") + + +.. _asyncupload: + +异步上传 +-------------------------- + +异步上传会先同步的将模型存储到 ``async_upload_tmp_folder`` 中，再异步的写入远端存储（OSS/NFS）中。从而避免存储ckpt阻塞训练过长时间。 + +config.ckpt 中相关的参数： + +- ``async_upload``: 是否开启异步上传。参数类型 ``bool/None``，默认为 ``False``。 + +- ``async_upload_tmp_folder``: 异步上传临时存储路径。参数类型 ``str/None``, 默认值为 ``/dev/shm/{JOB_NAME}_tmp_ckpt/``。 + +需要注意的是，异步上传功能仅在backend为boto3时才会有效果，bcakend为local时只支持同步存储。 + +``async_upload_tmp_folder`` 设置的的原则为尽量设置为计算节点的local目录，这样才可以获得最佳的异步上传速度，一般来说建议为 ``/dev/shm`` 或 ``/nvme`` 下的路径，如果使用同步上传，则该路径可不给。 + + +.. _snapshot: + +快照检查点 +-------------------------- + +快照检查点是一种特殊的检查点，其是为了减少模型因为训练崩溃（ECC error, NCCL error, .etc）等问题导致训练任务崩溃而损失的训练进度。其采用交替覆盖写的策略，所占用的存储大小为两个step的检查点所需的空间。配合上异步的检查点写入，在不影响训练速度和存储容量的条件下极大的增大了检查点的存储频率。 + +config.ckpt 中相关的参数： + +- ``oss_snapshot_freq``: 快照存储频率。参数类型 ``int/None``，默认为 ``50``。 + +``oss_snapshot_freq`` 可以根据模型每step时间酌情设置，一般快照频率在1小时以下，半小时以上为怡/不给（默认值是 ``checkpoint_every`` 的二分之一）。 + + +.. _autoresume: + +检查点自动恢复 +-------------------------- + +检查点自动加载功能的目的是在resume训练时，自动加载 ``save_ckpt_folder`` 路径下最新的检查点（包括snapshot检查点）。配合上自动重启机制，可以实现无人干预的任务自动恢复。 + +该功能默认开启，所以要注意如果需要加载 ``load_ckpt_folder`` 路径下的模型权重，要将 ``auto_resume`` 设置为 False，否则可能会产生预期外的行为。 + +config.ckpt 中相关的参数： + +- ``auto_resume``: 是否开启检查点自动恢复。参数类型 ``bool``，默认为 ``True``。 + +``auto_resume`` 如果为True，则尝试从 ``save_ckpt_folder`` 路径中自动加载最新的ckpt，如果找不到，则从step 0开始训练。如果为False，则尝试从 ``load_ckpt_folder`` 中加载模型参数。 + + +.. _stopfile: + +手动控制检查点存储 +-------------------------- + +在模型距离下一次检查点存储还有很长时间，这时如果希望立刻停止一个任务，又不希望丢失目前训练进度时可以使用手动控制检查点存储功能。通过向一个位于NFS上的 ``stop_file_path`` 文件中写入希望任务停止的step步数，Global Rank 0的进程会在每个step轮询该文件的值，如果发现有我们给出的停止step，则会进行一次广播通知所有的训练进程，约定各进程在训练到该step时存储一个检查点，并选择是否退出。 + + +config.ckpt 中相关的参数： + +- ``stop_file_path``：检查点存储控制文件的路径，参数类型 ``str/None``，默认为 ``None``，表示关闭该功能。 + +下面给出一个写入 ``stop_file_path`` 的例子： + +.. code-block:: bash + + # 我们希望停止的step步数 + # 如果存入的step>0，则任务会在存储ckpt后自动退出 + # 如果存入的step<0，则任务会在存储ckpt后会继续训练 + echo "999" > ./llm_alter/1006_pr.log + diff --git a/doc/imgs/ckpt_path_format_CN.png b/doc/imgs/ckpt_path_format_CN.png new file mode 100644 index 0000000000000000000000000000000000000000..0307d22f49ccdea557c611c79718d4b05e39e92c GIT binary patch literal 156419 zcmc$`XIxX;x;{#kqS6smN|Y`jSWu*+AWbPsZ$S}Jq_;>*3@)V!h*FgZ1}V~!4nj~V z3QF%?dPiy?A>oc0*IIk8`#)!&b3fdBKlmdvnRAS{m**YRFhhN$9>u77- zprT>~pJ^Kz4}fp%&MjnvKh$0~v@TQSxAM(XQNgHmG}MhB+AfWt3SCTsU^_mjTH;O* zDxJHb+c=W9LDy0nE6<#?aO9qVa$G3ewVu$OcKI}o)E>Iv#PE9&{9VD}%1*G)#y&)@ zlIpvUXp)WT&)LuceSL-M-*=u!RHqe** z$#6A(@Ig@TV1V)%zPj(#gHp@9DVW+LDry=!Mz;UShtXaAWtu&?YD8V>rqS;ca3|I@zOT(SqQo-C}a zOp40OOQ&sVxsOC5g&j`)L3!}}^}4|R5`mj=NEo~!k9OVpvYUkgh3E{XMAp~6_6vHD zq$b&91*5L`csuBF@nWXG$CnH^rKf2{XGBK+@OF0)k(QQj&3*Ee6G*CVi8+zgv3#{O z8Cu_$?t92pS_(JX-vaM({g(H}EzU+D4AIR_y1t?wr}WwAe6UiauQU%2PZtw|nP^30 z_En04uFBY{@YTHMU;+=O`q9jMnb}G@M%iYL{(DH;MU#`z+UE@b6VFs$inl*QzWQqE zsyt;S$#kdb^XEW?pW~m-U%YrR#>Z5GqHb9y>W!aSnsHjt^4Ga!1uqo!btgfjUX=!0 zm1eC&N=*%ew%UB9fu7my>3?vH)s@d7c&{k9uA|3ZV*5zghLAKYzB_!MUy@l$O6pln zt?O@{Hl6@$gk+zkkb57jk+GFXfb6GE9tdL6&_t9%?r)h(!r@^w(|H-|-zS?cb{3VE z`qrriw5Pq*i-@@IFs*j0vuTQfjT5YLeNM!O zVjvy|6Iq+2a|0o{-U|jBGu6C{f?P`0709^aiCAV}7|qKro&d|2_D#Kml1%kQg|aE} z2?<@F_bv}QyJYOxI7>e#t&D$qd>?uG0#Cof$)k2YTjl{hkDMdWt)#2|BD_oYX-A$x z+F^_Z78tMQ$xPw72JpO4ZB9#(^B-Vqx)(Pg3hfpE&o{>!Y!1hY%RV9T!|?@za5vNd zuA;Co=LyD&p?=js{^xkU*X`rzt*yMC1GgI$WDk0{57*-%Xfqe2!d~2qg3qG&AO2Lr?{)YF0o`_f ziW3houo$LD`R+BaragatCB&zpxSh&*SxBsp)O2UAq){9a5DBJzNpky2}bhc0=gg@8-c%yJ=SFeRUmrD~C*P@2z_QK995X zVn=cWIH})rK>M`c1MH?%8%$HIRs#r^UH#Nkh}~nrC7r$T@0B2>Khp-Zo&0+A)K#n+ ztzv*zaon2#@-7d}&cXZBb?bF5)F1Bpx=F~6vCQBtjZ!yMJV+ptbWTj%h8OkNH+2pU z`V*7;^IR2h?gcTtm5NPruRN_-DZmUFpDs~8$^1jP*pWZ?wI&|S>bKd zHm#OfbUI|=)Pu9}+PeM2&z6}@Z|+rp_oa_}o_r13^(F|d$%NCrYLNemgTN()p5lT& zC;9`E?nV?Hpa_5oD2GWnv-`+%al_ZH-aL(0YRhUz$)2iZ)bhM~doo<8>68(l*#_A@ zTW%P~fo1`Zl-W0`gDf*mMI|NOqqVT1U>FkT@0^DbE-Wl;&o_&!L&ev#O>2&vyj7=8 zv8e+f(=agYDnkPJ4g!h$5r2#06fr#q7tf!MQ3AdcFg?e&3Hb6z4*Kmr)jR>k_}*UM z+QH}rV~Yt~5V={`($YG&`-OQV%=FOLGVaibS$qD1%I?t%spszBa&gJ4F;N#5N_Ka5 zu)A@&L~M{-%Q>yE#5m72?%2c~3g66tZ)5MYtB}RlJqPA-g4#x(A}S!AU&G|och`u z8x7MAeXh>VCFO$RL-8Gpr(!6kdJ>qbuEBdTimBcNwCLM7Kmt;`37P7=#MbZN48lg! z`<$hZAkR|UpQKq|GoV{7p4@4?8eXuJkPtsaP0rRLo&9zexELMD${HG}F$?Ioae=8I^gnGd+C ztDszYMCR2_-SuiEiW#kT54u_IW}5dNn?E5n<*{_3yQG&rm7u zzw9i{6Wvgyg6h9^@UqmuwvSNf{n+j!4zPb?i1~x9X}$A?rNyQ8@`z%R&dEnO5D$+e z908dwkoYQVqsacjvNY`nAyaO1+tEt+z}icC63>w%gG`(bQLSZs=C=AE!6#IHzXXLQ zSu`-cJpD6*`s3rqA`tm+AMLgWR)l={kq7-;L?OPOMW0&;5HA1}{Uanmez?(JGK@y% zmus2jvJAb@Q5emTM~S5bzs(52NmY+IRGPswIxjtMndWvJx?BHP;_KIL`>VI2#AhpK zqst!e6{Uar6qmOi!0&M4Anqu^WJlllvN$GHG!4!hzEF!UX9hU{n{M4`1$^Q zj}y{Tr%kII2dXYbLskjZX;K&D1f!6-5|Ej`l%fc6eaHAB#6TQ{0WzJ4*lbYDe=`# zXDz(0_L`QDsOX&`{%7yL_haqsg*1~G?2kWs^5ksK{gmQj{?FEm7AH-)-6?*&3FH$& z>B?OtXen?qs{JuDZ4twAn8&x0!G6>Fxw z@B?}v)u+IZUa^K8SO4+RUzCIXWCd!ezi7K}pM7&h>2?m9bI^*>nSOUNS^i@1ep9rY z2$#1lCwxcih8o?k5Yiy--q(6%d)6uWY^kj=w;i9m`*$UUyx*j_+WP* zh^M!xUUar-|K6mY2YCj_MPD#Ik3r?A+*=w2G)aQI^NIl@J_1t%%Sc3WT7+NfxeO?xXe zu!!g;!6`9ZL{rQzvR4`s&8iK475< zpKueS&s|z7VwYshEh|52eou^zi=X{YwD2EKc6Un#7DlUJR+V=5&zOq$>#FcJ#Pmou zhu1%5?C-mDa{iX&?ycgWU0toA#n{IcZ1p^wVsG5t14JHG7zhj1zu1fMiOn-+n{DVLtr2&`YOA8hiG`Ak zQw$cS1r=^mBte&XM16~!D^lf0dgb>0JkvzY%MFBWhg;t6>8MLxD*dHU5W0Q@@|M?1 z$Q}&1O5=d8wB+JruK=-%QbPQ70o(#e=x8HKHcC6DS8Ux{4~*7biDr|1FKX&hs}Rll z#)jAO_-<$CjaMl{%+DOPtdGOn-JG3Q2pp;hYn!%;{MBN6{E&Ojq#b`nbr`DNiJE~` zV7L}O>$5zC24XMy*gcwU$E8DynQqFa$LJi=e7x0PRO;>Qk~b@_;%;||-`w`iv`79j zB*WxKyRsZ(zB4`>wuMD>pFuzhoCwaBi-rnn6L(K$!Ae?q!RcIboc8+)E>ma*BeK?I# zrma@Lrj25>v(+Ed>C?WRPbmG7D7-GPqB{C}$V*k%p68IfDbm+c&WHuAHCmLCm9F!Z zFF@Jfh$8(oft$^XNBYY(&^t6L0>AcrY1TB6!Uvr5jhA3$ z#~tLy`)RWxVv(V8Cr)DrmbfNX}8^C7Amuc_l6I;3}}0K zd61jiB@g~_y+kNV`*x@aHG$vCC%n0Q&;6p)DD09N3U8xO?KYg!k0PleC- zP$iW3yI_=@^+mq-0C0yxl9aft_|zJ1Yd6M0ukd)A{ast_#A>-WN|?HSn~J?qMprge zgfRKll;tZbEQS8~pcPYYZVByzk$$qEmi5?fu{KL+!J1WG-n7!hJvTWv$>SOmb&Xfw z!;rfx{g=)S9vc#`58BW<+aEYv9Uc=ek;2Q{%I4>CGe@%j7-vF%u z_AT$)1K}0_l6RlgpY*LYAdQ06aHOkPogpg@snZ#|Jn=N456K-pY0Gzl_Y!t23Qw$X z>rf9U3mL)6PKHjQpENx2-n_Smze~hheWTg9?|1(_e_>l~FB}dx03Adrs`i7Yos_*) zj+rT3lK$Bfi?={({bnF0*&)F{A;&*M>X%Q-K#u=&L~CYxCT#>0oQj48&YIF#w5wzv zg_Fy1od)k~{y;givcVS};cJKsA~YK<4DschwBfHR{g=!G79U|{)`~+iy-Lx}`;*ht z=_Qp#1GNac>B;1*E8TL)K0gh9p&SX*%TU$A7n&k{T8d>@G}XEf_#E31lZGOtA2ztHL*G$OQ4>23;!y#O8l-ZABw=oh!6pXNY1(j7Wy^ncSoHFtZz_^u>QI9ScxSfBZ^LV2nILF= z>G{koPp4O#>ewhU)vM?64^zp!g1pSqtI*gXE(j6-p?;SuYO!hU%%ia1^=LK75 zcKV!vvD|^e#M?|tqd0K|hJr8b3;J_FRShL0`!kE)w9C>!o9E+dXn@fT4i7W^a=acX zvLVzi_=(ioL-%3KQJpBY16nib(hC<4wyho_BV3QOIcRPhnT7yQ?{;!2!)-^S3hH$a zL_`b`2408&>g!9jqVvnz!u|l|*6#NH=&*AsCq0&5k|BP`Wu*UE*~%S;J3}1xxahK% zoK@v=iQiq#B%$QOn+_tKi(8ky3g7hC8%Ln@A_5urg$MQ7(%`C5h+^uFcO)P{^9Ln( zzm~i0SY8&#K|cuZ#1+;f=~^Pn4%}mCO-LC0w9EJkuSf{mO~IY!U@r^a&-0cw6}{J& z9uN^Mr7h?a>SaG4*`Ez7JMcA1T+H-rsOOQH*{-XnKfU#e_(ML^J??SClai8(5{x06 zLsO+iCB{nP`$~JKC**|%pjlXCt%~t>R7oE7$ajFKrxcAW$ApjuAe>oHiZsVt$q$48 zBwRHm`&w$K=aA9JktlF5u+$XTqbfnfNrC&w+|u0puvAz?veJPP-2tnZqGriEBjP*nwKJaXBFUl5RjSQbKt`= z1Hpx#&4nc;4^_r-cp~(HzWh%+OSqL=-$hen^Gk)r;p!Hwv+Pj+H3d5R0Jn5|mE>fP zY*9$u|Gj{Ehw zk8tqd5cRIMsC?Bc`U$f#+*HBfA5!-O;fuiC5-s!c6zY2qV0=PNXiuJitsLKvnkJ>W z4tVY}n6Y^GoETz;aj>kzcbZzHUNB;9Vy4)%3i4=uUvOV2Tjat=66zOd_4M@alb=W* zF*S>J5n-MxQ_OOjpg^Gga1c&Zl)nKa4EoM@ zkvdzDPLP0a(086$8TQ(vsW#<$NZw78sO9<|KNy`F*wL|j<7>eI!}LLmcI2!cdDr_q z2fbxp$fCMH@0El*xM%*FG+zb>EvrX?-*&C9w=OW6C?!*X{+FORQ5v=7 zZwiz@I%y;g84`dy0nx}vWjxfa3scPf<#~agC%}-16enKk>-QR4V*-Yh<(aedN;Pzf&4>YwMGHEgPvgQ8)(6s0P%5HfXvq$)tE zij;)(-h&_S#X}PB&t=akgDfeUVo7ge(z}fg9*PrxE?t`9Lf`&i3!U4^ZWtX$yRJpI zY_BtUR&RTiZlk>H)YtF`6Pk%oDrEAxi>hO%WtyYXymqH{R;7Nz%A`V61Shpv%~<#E z-knnEMEPn+L_(qWca?F;v*#7KO5>(f(TdpE3#ZXD_&<1dTaMy@h!}hI@6V3O}m52MV?{4`T}Ge;zn~{}TCT z0B|cd-bfIn&qwK=kadN~y8Yx)T8`DwflbAUMY8kJ3GDQqB5taUVQ$iaW7;k6SYc_6 zn(rMwK2TB|Il^2n_w_^kL~Rj`dfAqGnQAkdwq<#lSz)N%Hgf!r$M$)}Zx+?mH8-OzZ9PaEV0Z{@H(UcPj%e@PFXE_cT$>!yIkbCR zi>B^IF09?Dbcmgafx+xH5p8MWO@Z;gX8|c+RkWpyw^H?h6x1kEK*xe1BQ=x)0S(7! z=`u7^-LPugar?u=)~`}h^opGKJ3Bk`smL2zWdEPH&=f4 z)7qrf5RO?{N;ny7PugyeoX6&D4BAlk9WO4EkKR31VSp zfk9~~@D#di;8r=WD=9dB1{g@n2L)wo7E82WG=f>xqS`z69z0mQ4?00#HWgFa&_{t7 zdob%#WP%0n8R_$p;-$lg4K-kdGQ>plf@&eFnlb?Se}s&edK{r6h%(30huR@?PqE27 z{c{2_pjX{$rHWL_%g@h9!rPbIq^GR(BSX6-U1{6|vT!VDV0%aeMH& zk}?g-&;?wWjJ!Vs8J!`3n$0;|p=t;wYXY3~{epgE(q(jvFtdi5YT4n?X_BHDE$-e^x--r)Xlt*(49kL+<)tL-WoQuEGIV63XdsC?x0uf0Fl!yKW~B z?FuTJGHR`xxyK5@*wiwwyYz{g=?}&|Ee9D5zgO>*OdS%Wu}XjQoQvkJ8;&t zSxac~#3Nw6lFfG%!>yAAEZy3)*0cLab485VxzU#fOgxwAzywsA>#JZygpBY3sneDx z>^_p-dqYq6i-D(AK2<~2Ipeih;Ck4qD-IApb}1k%6X{l=1PhZj0_%n?u*97(6gRwN z1QS?_oNT%&#qAkrZ*TUt=(gHk@#g1n>Y~y|zoW@D2#X)|Hx7VSN{s^SN4X!bXWamNW`Ea!>UP$ zia~WDbdRX18n&+9_qy>lY@G%oy7{l~DMC}bP8~)g_tP4j%t42-KNpuhWApappRA)T z!I1!`qU9bQ7EzOZ%ipzG4q<7e9N-h9fv$PsEBFr`_v--$1Kuy&g5|(bC=K*q#{&R2g$O{( zc5785Mwq!<&5P;Z1PY=fG-VB2J;@DV@;WN8$kV^~8-O^d+6Pp47lMkO%40##9zPP183OKVJoVDiV@{gl#utg zOyz*~y8rMx3YHhW3fYJElkmUP>jboy)lg=}qr=rNc^?nu&cAs?*8m0!69!kn0VK2s zqc#`spC_ZBdjiA&DXX~;hIO!NhtDVcwI0Q8>gplo?wr1%4j4Mb_FUX9^snDvNrT6* zM1%iN%ro&8dX)rAcYHHMBR@GUf$>~FWPgt^3OMOMKKlw0^OyGiW$LmW6mp&oPeIg+&MVRH zHpf%kMt+8g&2}EmEbo*MYfIli<;B_JkF{ODyT5r(LV4}W${Ts&caN=So{IDftzjmd z^b>v$7cvR`bH^_O4YNI*#2I7jb|Odqd8%>S7W_E+x*CGLu1Y@FyrGt!z=#vyEq>ceD$s%8C#0re)ChY2g8{o z${e_0aHDFAB}nPZmAsM7meJRp2!XMq44KK@x{fJvor(s$U_dqJ=BeP*q9mP%{F?s^yM1jc(^BB&*BJ1Mh^ zHD!<3jOW)3O4<8_@A7I}k@EAJ&=E`Tg0E#a`^Uom^PWsFi-N=2VpfkwHQ;3KA*Cj^0P>9rz;^Tr$^fHP*nNxwPjc)!yLGW~y zfS?sY^=HiD|I}lh9#+5L206MK$Y4 z*m#PkKgf4Zzp0;ZT0Io~&bup55idUY!lAmJU4%KANH|w@%CF%k{@s_=s%*;!FUQ%I zk6UFj>F3(~UB=D|qLyU3dRA+&+d*)z-hzw-+@8t&=Llqt=ktBUsFxo>CVGEsyJmkN z3HL%}qq5qmPvkT>kjRzt`+VPD+}m9Sk@ecWrYr~!*99?8{JP1u#;5rYHzrjL;LpY? zn)%lyX=4v>?@#ziC~sIJ)bicxa&^euYD8qEqK*la15lS!k3=qAFy=~mxPYa zy&?KGBS(@XYRFIGH=DDqV65i&itv-jhEfL`2(K1A)lE#j^ue`Q`1 zK-m3X)UFMqVvw`17Ih1r0WB=ec?3$4uU{Dt+7%@b`DrBrIbmG2V^yiU|LSZzJEXuMKG*saS1Y zw_>lCOQJh_udxpmyYIcS)p|L@v=Ra$YK}f~{tPcdifDcN4 zQ$bSl%ATEr^a~M#-a@!S@qQcMqxh_A!pD?$epxF_PnlMYCp)3bVlHOfMPnw{qQ@glp|(X5BoPwI#HFtJZ@B&H?KaeUS7s+-e+;LqN*strV9YL9 z03UhyCLkTcSN^gS2xtHau~wz$Bsg(#9o+FX!1`Yy>nie=Q6VuVz;fOpGiuM2)5>o4 zVJ-)6oOmJ1=9H10(sl~rWvpvW+NCg>w`@Z!*4_=K82Hw@DPbz)fSXXeQx(D z0!ST8=0DcMX^KwPB{|lVb*BdI72&R>kh`0@P^6z@2+aPgVx(LUp-hn&f5Ir0tLWuF zMEywZg}5CXRAkMAEO>SFad4NASe{L4ZPxXR>j&wpZ@v(hUB4x_O(VkmETkeVjOGM; z4xQ4@tORK-q`AQ9#^y@(+Cpsp$K2s;8dI+RqmNAh-k9x};-^4r4H&JJv>3Nf+3{AG z9lp55Hw=o%jbX`u9)8a-0HWF4dVho+Ci{Rw!sI_l5MlNfWB#}zm*+j@HJvyR=XI>f z0xCvd-MGm~zu@%o=MS&>_bJQPHq*Na;)Ro@az?QHy>A9@bQ5d>ZO!^@Jm-fxiR7&< z!H2e8nBft*-F260T3gNKFcnnxBtKhU<3@_=)wMnEsIA;RY(U)0C6yd6B~vD@CldTgibNWyDT+eZV71^1NKb*W30>O^N)*)5@<#9B>@;wghZ! z?bg=HY}#BUUmV<4E}%_i4L_NLVy#i03By#|dft+>SuY4D`f|+F5rn)Jn~g+(nSM7_ z@U)5^1Tk*lIwEkbcwP^&B=ys?Q%5Y5C_vC}+Yn(sq&Q&%29p04r;AU55NGOdsmR2Z zXp3eBNbtjtlP&D$=UBFVCXzjG==OaqHc@~jg+xB;SH@>-jthCs&pQ%}n`%@7T_k&$ z%6#MbJ}q0-_Ty|kkOO5sVCD^Lw<=g*d6OJ?TXFu%<5A~t`K9a4&W{5f>i>q?*_~~ID)Qw?RtH_qbVR7S0TFMwWV18 zy%8hcKG3&Y=a%HbZNKo#Af-~QP=DczZ`GB*XL_rAcnP$mICrzL{+Lln1EmmBgne-y-3dg-2tAKQs{ ztu_)(SVCac2FO2DwWc@2+nQgPhBw|JhtK7x5)5?h5Y^^l{ z0UyBIS{H+tGcco zUiELv`}4h@^ERm(ST#EOu^^jDwQG&+)Dben)+*wBpW3Y41o`MrPej%I5_8KPWoI`I zl}T2_IZ55rrE6A2dF9OZz4zgyS8?xJp<1qiJH-*2FYIBZAHR@P7~)ypNju&gm%_S! zM}2Tdcki*FG-|A3HbK=&+K|VGybxrCmk6j&B&D&I4u(l%T9gwG%g5CiIKmy7xMJFW zeTp_YZf0Ul5Mkzue48u%xYmw!^0lkPSc1_?s(&973(IS9HZqS*hq7t&Hu2ATthIE_ z1W!Vk2o}^dwjo8CHE24tTqX7oIk<$*&UIdKNC{opz{+H|@*90leTO|o{WI_fL*&5T zE)gTnM)GY*F%CGrxNY$)`fS);%+-PF?O^I0rXD|@J9vqWJyVR(hYzZO*Gct_9}F~w zTlPBXN7`(Ky{-K$qNV9P>HDs8J=|ZlOHXB>aY&@X@=4G&_-z*3d8B-x{vL>} zQ#|dGu3B*+U0t6vD3HTe@5Hq$WO^dhzkxjp17C2QoiR(ClSl;$widFelM8+RG z&%jy*#~{>uSJYRRNxWkwgm?EeW3^U}Dy_P<;Z`G;ACxtEj>R!5lnLx?r=gONi>R90 zW^!&He^z-}T$j7hwK&kUA*{ z6__6F^QEE7xMBw|9!DXedk??8jb*`=21x8NR*iYw5u@t$`;m$pkM@cPo?;JGB_`Yi z<=l!1)5%00co+h?b=)DAiSF)J_}SC`0c)L0eILC}V^FSm`1tJ5#W~5z>@XTl^*X5? z#e2`WZH|*Re?CAAY0A#!;D4lBk>4pkIzs$``Z+^}UzmBU#Ll;39d_bczMq*SW_tqT z@I@#!VIQlynBz8jb47C5_ERI{her9Oy>K~#+_oBSI>JktQ>pn`lllw>*M|2Y5ArsJ z$Sm42e;nJmkukOr^v0q-oMV7pHG))|raH0HVQ^Dna=Jn91U+2tuQI3$7)g20poWs!d;sh?zXI@okiI7w=ED>1fLke=8K+pI}`= z9O0iFM{C_6e9i34e}^{bzT0m0%V?FP%k?$-{mJUwptUzY#MX?HI3sA5&qUW(FTRNI zZNtZyb20s~o0f8awi=e-g1drkn;ei)2p?qv7LP)HZA5gFdJ6g_Jd7u+mo@}uxohAp zA7>R;J3!yYlP`aC zE}S^*Hp;x9X}9%?XFFk)!P>jvtlsWbyXic$W!obvgZL)F1)Rxc5E0zFtO=jR2r_+e z5u?<54v#-jH)BcNLUz7WW#r}1fUAZj8xMxm;re*uF>gbvH5**HV%GV&#jZJ`Efc>3 zq0;QvJwuxumF^mFfP0R6=+UMwrnuN;pjoN7(K3gjOo2+I-E`P(oZ2%K`Q%J>6lAXePa_njhTCiiz2an}YE@VU`c%VKJ&yEGyk^v!g4&xK-B zuOy8-?==MRk%G;t$?JQ{BgD1@radltzLo6Xnr{m3h=ccUUpue%pergA_}#S? z2E4;;w^>=0d>}}~>S8~?Pu@#$w!Z0?#wXuT)G8vx)IY9GnETwCJve`AGIz=RTe;Ai zEh0i?Y1!(UO?6=PgeU-wP}rNMG1yth_UuC*E*)N0cUiI~5`%P5^KaM9dThu+s+{!A zwf0xu-s6tk@x#%*!QLsZm@TjERW%a7DWltee3RUqFRf^>z4&vFn6t3cDi&Q&{P`L# zshXkC_N>QwA~eodASn0Y@sNi&%VgIcHM#@hSQ|$@q-r z6oUY49SM(A4%$n*=zZzXpf5~6Ufgi?k~?&^^lzavet!cx`+-F-X;C%1LAnC~>}5)H z$f3Ps{AF)CKy?KR)5OWdh8zx$T_${>em+*4ub5~!EjfcNi`^Y8FFw4BB)n$ucVhqu zSQbZJc)(17AlLJi(vJ@Z>YRF8KLi<-xoyl&w+M1c$y6EC3LXf$KPqXI>Lgp1m8_ma z&{k?O|EzNnb7pp_98t7V`Abu-Wjianen~|GnRx;9mX25Ys)0U*T zs?%p1WMa0~^-BELkC6S8%{U=M`l`ydS+zi1bkIcCe6knfg-G}tkuVyQ(fn3$KI-pu z1&#@VSLS3}Y6`$X?`t7-K*rghX4SsMKUtwWMdW_K-Hny8CH9m%v8C(UD6~m)QoTR+ zsHIN?=XnJ)ez%r|M5D2*wl7X|mK0q7g>c8#fq->YZSqMI$KB(OLQYnQtC>2;h2_@l z%HUd%OvCeWtQ^}1*?O6L9r4-KST$?-<}jl7+C0`I-F=G*QM;=sEVb;0*RHSgwZ%~-Y0CC;WQ%m`lIn*%WG|=6K!H2 z8HwI<>PI*l1z+x|4|i{gUd;}Zehi|T(fYZo!NuqYebYmfPK&7`Pmwk= z4N6hAv3$>M+`px}>8%p``d9;lzz9KmitgWS7mQJXljk;u;X@#tJ~ALbz`TKeW)IjuX_})kK#eNteQ9 z?x)${yvf%7{)<<-wvEJ+GIc-bVC)d4<+nKKvCsA%HpjQOiQSK7a%VZlS%ch1gSv`H z#*Ev>Io3wmAf0=4=)%gT9`BP(qrNh{SrnG-*^>6M!Qjo~E@c$6VDP72N}XBaZCkO4 z%{7X3H{7b}YP+~_4!yKjR%{lWWb)%IPWSXy+k@t6n`O9yb}DI5QR>GW_gRtW#5^KWhr zL*y!h zaF+P@@Z>HTDvli3Ua!dr^`HWE)SmJJo!`c%CgfHYr`_V_Dcz_!o@YlJUXULXu+FG$ zMYn!CGgIHVR9B<+ifgbB#r&E%9>z>j4h|+d7v6I&wXU3VW|IrXhMZm{pH-pRx;XM- zL#mfFQ9HI%F)6wBCS3Ze<-C^ah@Y9s;Z(CyNd=tl?a!%><@Y{v)4$Q$8Wo(eh^(0$ z+GuBg=)4NR_j%VFi8IE2Xd2pz)BD!d+ zp0%tMR90o=-B!fG+8qN4e)5l4(59A}9LEl~E;CHv3=~V_T2&nv$3kQRe%f--Ggr?; zl?DrLBino7eYM%ixt;=mq4+IU#sACKK|H;yQx{HC+&wg?3knWqNWQ68x#uDu&0LAD zhu61JW*cxd9v^Moqj_0$*eT4|q6&Mi|DuW4a_VK@`o2o57RChJ6{F`5`bhftz;_9QYF!!XO$%w%l zu~de4^?4SJgAf#!i?Uc^jvW^q87RLK{z1Z;%YO#Yq|Xf6D0e+y7-=A zOqCAHq^|QbPr-+vAgz^8ctfzj&s9%-URWKzg4a; z{U9sps>1x#%~Lo_VP<1duir=2r>#13E6fOcrO``XP4bj!V`$GW+&RLBSYa@0|~(#GHv+ zFCT*r_t{xSYv1JP^@Cfd8rh#m#vK<3tWadixpvfn5M;?oA6_=GI~3BKwXBlas(3#U zF^hb*>VMYCjN665$fq~g4(ix4BH!9DJpn)0n<{>s+&Eg790DN&Z=~l3j!^+w0^&BOc zG~SltSK{qCBPMTfXD3f8immXpJl6_tKi9Ubv|j|@O>mQh-{|S#-4=STun@q~8AgYW z9`00qD`n~)S~;;)eDaL_Y2a`-Zpeu}#OxFwv>kxq!IQsOfliu?aU^eCEbpYBjhD2* zs+HdqjsvlIB9H!de%S8B(z%=d1Mf%rdYGaTF!t-p1AYZ(R||~jaV@oi3+9{0@*K)D z=RP76$M>>wKM)SxOz<>t*Ztz`@vH7j-sZ~UOu5@u{FS^-2D7Gk&C01SOxL-@nRit8 zmcu(SQPu@gAZC5Gi9E`a3{5pw$juc?H|k9u zKYhGlHG}7%50A@!wt;L786B;0CfK?uyAMWK&4cpv9LGY+x$hPHns$YKy@DOseva`` zMXPF>mMyDlVlyL`ugV6A!)x8ltn~O3Ge`dBXB*f&hGx32ImfgN3E1*g{hd(!Y+SwqMYr@VA%BTocxf?QDL~RNi{Y#Xk?jtVv!V)KS%bw^a8W4W1YsE;Bak2 zM~+=S(uL2K6#mJq?J`!aqj>!yt8*m2)xO^`yEvCYAc|X{*@ZUN#%vmsusWcCUz7vQ z?#HwF+*cP~AzT&P5k46}JkJ!g$vvQsZ5#I7@a0K{hW0eJDjp#*;$UD{I@$4^P;p8_nv{+-RK<`4#rlg zA&|7P_e98l&1D~*Fy*M&&mU2bv?ZB|7>)C^Es}$Ax~ciiJM(ekhMV*rD()s1%#(0U z6*R{!s_|!y>=H#@ZII4ilg+-C5{=!<&aruR5@yA<$KTX9hIwl8cm&hed$>^Z=$SVx zH8lZAg#P1uDaj=PCea?MT9{>VqMS#`+iaVGab(gN+mZ=AqfYfWx}vjqX@$(MZ*^(v zrG%NMhIqXe%4=%_tX^Z0c|;cR*U9YznIL>cG$KYE%jeo8O&X2kimdU&J| z2VjOZFE|zsY_7Y^6XSjs9@dpucEG9`;}5&N(K8@9XHJ*|XJdZwi}IR0b>W(xed#b! z%l@@>wct&UBNsrn8doby;>hEJXY=wA`aO1tS(}9e{n%$8-5gV;_ossfOq3Ux-gOT| z?QoFY1W%dW;I9t6Z9Ia|AP0>WjPB0*c3W9aHGKjIs4jlYq*b={)i!wFF5GX@s;b;! zuCzhZ=(6buLM<-u=7-T^+apPW4+75AwnUdqoLj7I+Ey?cj|suWxbD>8edH$o8s+tz zl=mK8>@03mDGqWO0HzO6Oe2+cK~=@|(h%sNL5cBkAhX+^9RKK9*|a(657B^5;Jl4p zJ>U!-j)bq6F>Ue~nN`-_yR-sc$L(1f3dEdD^qDFfwUU^kO=8d8n^7j2nTR{-;l@3G zWH`gi`oNRlM12M?I?Fme9};gyO*V|ctMLRH`J9$44*K;*hsF_DG!j<;O4jC_<901# z6VA)4BP1z+uj!8M|7^_`buin?vhNs(Iwkq4yXd&)ZDS+ZO0)4xO5tyYs^IT=c3Fr5 zkd&|{G~s1oU!6DXar(ElI{Kwv{<^qS?ZxG#(Xo^N+TCy_$WZL<$>xzo4Rp%h)O+uN zhFKpuO5q7XTn?`WeDpi|NabUG1_~tZf@9{rvVioy+L>JV^_1G>Rx~~mvaJ6@N}x4@ zUKB`OqY={Q5+^8B!BbBPO3fi$N2}&!c8T=q6MGk}higc6^0l!1V^m@OHUK!}*(3#f z{50&2_c$OQa=6{(u=f#Z2Ia59sw&~84|siUYxmT)_7#a!smvYaH33oR#+naK{Pc!E zuEGI@mfY9k&so27515V+-d?i`z;qoQ<^%H=qv~vxD<-u2nr+r2J!;{HHdfzeFTdx^ z%6svw*J8G{Dqbf(OA*Y|j)=@3wk*Yp7o{#JuIH^nLn8!-OZW?q=1qql;*lC$tLnvc znc|Lv5w=X8r68<(brS@#$w$Qf1zKv|6glY$qGvbz{m17sQ}i>FJ#}xT z?+>A5TwAtq0VafAAyn^Hi{K4jC-|L=h&Nb&XuQlhoqqq;RYoVNsn0&X+`1qQhI!rY2gn5owYK}Iv*^KBZ z2xT#n5O@`^dQ`q9r6Du{i%jW*Xl|j4vTfij-W3=WRwZg{ZjFkS33roeVh7qIq0*B}(y(SnF_E>0K*5 zCg=qF3;E2kPVUAOtD3I$Ibb zbc`nW5^AMQm_kaJQN09tT9y`l$bY)eB{oz&Tpj(0-sN@@H#=tV!unFuLV7JTJIij1|2%B*5>)eG zsz~56LlTJ-rs^6m+a)I`Urem<&5c=Wj_kFLRlbF?8hBTy17^C|21j$Yp-ZAX)y$Kl zD9q^BI&&~mU1F%qXOOAXu5Qijr03;_s9f^Z^nAJt=+(KWY&Ky!+rNLRX1&YKos{F_x)S(DI!EV z%~SX{*%1vAnixlUHGCFc;aUd za4^k|F{8y&Rw6|oG%P7N?u#Z|C+=h&u4?bNy1iW9SOW$Tc*S_8%jHf!_3-t6F?EnZ zb~3q}+3pF+qQ;RE3+U8y)nch(%q;6?ICKp{fmO`jb!3YeQ* zclLijYp+3?uTH2T!Mj()d!0~dbi7DVCF8za6HR_`P;)rnTIhiP!hw+PEunc$WJA@Q z^+X@fqkHMrTJDXEQHxJV=atPnbgq+V8Y3_-~_V_3k*TZ!~&nfDhh%aBU zTpb;DT*DTo1b4u3DEW}rXUp97ZV}0vB-+bOd(+&RQS!XJiJ;PII7}SAi zd(7(bjXA$@zpDTG(A+T#K;OyOI(RMm-_mv2g67xxqfAYBi)~LtzXA8d)(UY0g-gM0 z=QHqp#Z$g~smjBW1$tD*@t9S>NCCkOl#-mG7^7A^!B?gJnQrTP>+MPk#{qmR z@k8*%TOxz54=YPJG6fJkgAhJN%B~MOuO4b9UGLY+&Xm~#*1bUuj+0s?x7RMk9gIj1 z7V2g3pA)5P^LZG>_;3iX_kB4)+}Lz9fSfcyjt$n68cg%Iy9Iin=~@l0&TdYEIgj=P z0%=ReztHn{9SXaT#&1pv+fJ7N5w6n1R6U9%J-)#>FJEXr_>#AA&p`D#{`c*EmtYfd;?&kyQeOS&M(8R+FMSy^JUH{Ra^-6H~lFseC z#(AqagXa2Y(G~89Q6~dtbgpHSE%_CU0PuF=b~ne&{gt}NW`OVzw0S26 z)M5yscnJhSCe*4`ukjnf2E|)89&c1U^q^wi18tFy&k4~}(Mvjp&gV`o_MV_N5Tc>j z2S^j52V=4BU}MLezT__oPI)$3DMm`;c?WDKC^U?ff*KOV_sHVKCS#x>Z z*veShkrVZ4?9?ch>1ARN#9C86I8@9Fjr>f)`S|*(QM0OM&wpbmW~bWqd<9VdzuHci zuFvM`m@Z>Gu4`o~rdHY=keWOensUn{g&d+3!Pz=a;$GK(Vy!!spxS?_$l*T`e`kO$ zL{0g{dPt*Sf9%%KG#T9HjZSiFvb9!Pi@ft|i{Q8|R2^F$d7GVs;v98roLt%*01FGD7wDo=Ok9$)3^2t`%R(f^cvLgSm9k zzL*u39;RF8*Ow7+al%Nd^61=`7?|71L;j@VnlH|t=C45!P+w_bb4o^(I`n5`@bXY* zK-ZRZBXJPq6v5_0kfj@_OOkCY4uQ$cSXJ69 zJF1H84COZ1{@f4+3LIa1h<~nfEp=?JR3lxdH9dS8*QCAc3<@~4{xzGFXz58 zi@q*<7UP}=aO+ajhDRL}^e{wls&O=5>l6#VmCl)H)Ae9VkQuo2L^7}QV%O86!$L`i zuFcUJXMrCs_?Q3lx=R2^ zlHccr%BBjjD5l6_(HP^Mw-`xowvcNJz|N@~WNWPDkZ(6$zLSx^(0RIaoG&?j?jr9{ zwG|x&?@zFF3z;mZIRoW@D?`Kmo324l?aY-hde$g1w%o0dVsE|VO|uW1MifUNK9-ZN z9B}jrFgcU|a!+@i?a;WEe(3T^%&P#pw{+-(vs2E3|7qQ|H_;AK z&qEGQ!;X8A!ZsVnOtOE(7sG5^iF+$Pk1}3uhR>y?_PmE^%(ZtxzeVQB}^YQJ?5srU( zqMk_Li8R*D&FSDr_r3*H%M89R@N!t#HrMe+cUIriW7&nq=Dm{Rs+lQMKendD>`6<1 zlC9z>?#1B%s=m6Sw-PKs&4UE|Shl}VFuH6yxa;rUNS#X1kka$Q!*ojuUcVh}yN;D# zws|>{E^C5Bhbe2)u`OB#`m~VFNGu*G>fFkC|L?kXBL>|(JDMr*yOY2>kitD#lK-f( z7@eBE`&3{lBl>E&CDB|sFLwF8G@L){Y#1)6y}YSjpMG68RLox%6FU7P{pzMq%}^Y0 zl@+W2y~p?Y{q2!K2yFobIDNIP3j9EVOn4g&$ZWl+fpqQSL|(^p(L>NkL2xM$oqG|D*{7-qgee>W766&ddZ&FB&K^ z>;{SWg`0A^CyXkGCXCES*WuWHE%mY>)g=tH%{AxH8s#SxgZ2|;!I`%Q>0#N2t9mm# z!7feZp8-R+ootRJ4La0Xs6oF{o#fKLGy>1Zvw%|Ktl*t_&26Ba=+mbng@|QP<87>! zyyX~TDaa9PQCm@c=Z{tx_pyOXNqBWT;rlYRb_0+V^M?^LK&Ic>Giy3&P!It5pgO=_ z{zn~VLr=jp_HL9(=j>HcJX;H;*{j&?n_W<31!px6@TcechKmF2r zan~~I@JyRLg$f7~5DcGC@|b^XDfEg1+IsuDXPkfrJf?1~h4gC`~`0DUrSjbyHxa2JVQ&WPkYEX9sR2%AUros$Yjw9S0wiS8dxBZKt z&jB4?Df9{OgDKhA;YOduta5WOdzR=tysu}?sA_V5eOMXvC9E08zK|^eGGS~dME@VO z{n!2f=@)RgIhe?CQ~1$r|K=&^l2+9F5FiX!jAKU*Mh5k?X{#!5M zpEn5dn=$*d7U1tw^Vg#rrvahqVR3x_REG{A-XnMxe)_hOEAS{pnBn{{1h1Tl)VX!QTdX4w@$i+O(97{>@(gl;Cgb_*=W{ zf7B;9lMgr;-Qrx%zm4+m5BJCS|7MhbkM{rD*MtZ_9O3%&{QQ5G_xCOT*6R8X68vkB z|DT!WKhDm7&GB!>_b=7`X?y=Q$NyyE|BpQ7f6ei)8ThYv{EuPb|CPY~moonT%YVJ& zf4$@XHUs}P$Gv7y;&B4Vba&E& zir(ap%JfrQUQf6XTjk~*TwN!g_aL4=HfN%wzs^7`HbMUNC`5;#Sm&iUQkyn2=G%k( zu5YvDXOH^7y?QraB2~z!K>1bZP<3aB6r@&+`puejCM_Xm z#nLBU#+&;f;u}9foNP5DFmRz2f`FeyDNk4lfm(XDUr$q{f#U_*?1|?&2plJ zchdjis;!_FCdzc#dRN@3t1!nzfz8PV>3xC_@$^VpdNXmPQ&+7_QXwxk%7{R$42YY_ zf@7zc$ge$hwf$cizXz?20u$y~#B3xEmy32j1Ys+cTdUyt&8LAxWMiBv-E%QHLoyvB@M42>tbY@TcS@wlEzEF_Z zSNyOJI{Z+nQWZj1{}ISpnF5)@reJHGTZ)Kx;HWmlRbWYkMOVF!sDX@npw z=mX@no^4FOl?XbSVI>=}p9pAglxPK}e*lH{1=Y=`RcAi!={enRGhzT^Zk6BLV!s3A)w>S~NmGA_Y2(){ z9d)=|KHE_CH2{ zMUU=&^2Yk@n8CAHXlnEY-xypo7UHBRYq@&t&8L)C7z5-E8(mK_Y_A))xz2wk1{aAq@9Iq?m## z&xx@3X>Mczr-%d5CcJHHe@}KL_*ldK+vcQ%d|}~rf0{k`J6b(nib6AKab=FV-2D_Z zWFx_=VDWVPU~xJ#0y+fyd_M@@UEL@Lp{gKU+6FiOg{Fv{X5&{GtTWkZq&g3c>t=Z- zA+%`1zioC9jKuGJ%Jc5G3!n$f$P8cViRgd0`dxVBmLKGP_4qekX>1sb+-6>q_H!V& zELKXkMxqjBLBi<~HRh-DmLq6xhFH3yr0$xm#FvSLJRdt0+(#vUnl>}Au1asFso&mB2rNX$i zh_Vo|Jwut4mH;EDLoZa5s!feA#8hlkiQ?T|2Dp05!7B>LrqcW;%$}i>HBQ-6P%UO9DR2ej{&RzzF)5uthae zH&O5gM{n>`8$nQ|Zcoc^(aHsZ;ru8Zl)SFCH!lOPDFYhzTE?G0A6nkmH~p>OL}{>? zIL8VJB%Y@V#TO{iN5#lK$rm}|Puw4C{U(YihhBBmJKVFhKNKsMuPsi4x0V~_p2C_K z@wzV%J7v8VmyR%4N-g}(#Rk2maQyrVo0N_fFp8A2A1>h-C^i!X`nLX1--RZv5z>Z0 zt29DPf2f@@XwEE2$3;jH`-&(118>%V<8J2)Gi)G)5zztH+|=ch6j#zSBXil%kH1Yk z23W=`X*FLFuOIsc8}R2IcnERTo_rW-7V!H&qTq{Qf4$;*`~jh{&*QJplX{m%I^B50 zl+gDmN(GmS1710?`KqWO@JLPHIfPq2FROwT6aE@jIVWmzHm3$1z9%QLpo{f|DIU^f z>o_He4sa*NG!R0}@S z+Zi|9!TPJ3?eHovvNdl+_36{b7bYmRS=IZVs2xQ~2Z^UA8G0*62a===W6u9#`}H&+ znj%A{T&F&ViE-tbJk!JghR>;v7Nb-;Zm<(+e0ER?2S&K3BM{-8(UWksTllF$PhQ>s zhWfq1_k??$fo^?}{0^=v<JJ>?wN_n6y7E~Y;cp%G9dIl z>bPDwwxVsO+EMv{d(DHQ!Gwin^@!|@?m_M|QGBJ-k1+_%Sio!|=BTIasJ*}z9`Ia~frPX43?45=jk=>*) zw$FA(I1%p6Td(;{^=!(2OVSI+9@+`a7L6BW^KmAUaH899k6ca)7M8=S zh;4ElKgjJL#C@0-Mgl0GSP4~lq?ULB`LfYVOtoef8{Cw52+t@?P{S<{`t045N(VdL zY|ju{ttprSS1+3M z@PsKm#afvhBv)b_uyDN70i5rE3Gzv!%2SGhU*|(|Q%k?yA{V`oiA{84tLdTFE4sDEK^~6ibAlRd{-oMc@}3f353KD#c-{ZDJ53dq-%0t zDociSo3A5srK|*82dsMi{OFdtB2GJS55DC5P4i!xBc&p2QpvTGGORJfaGQJ{+1lL*{fu6YjC8F$cgXY3UFB4hm1u)g4r z@F4(PNJzyhs*6;r(D}3)83snU&4BBk19UbpjsAuokJMZB(3 zrt1YJg&0%hA;0Ft(sXgC2tA>Aq`wi4^@g_yj%?BO<@eya2;mdtK6k#s*M^(TX z<=wg1+Noy+H*h*p_D24wW6ShXWk#z&1(w*TMmvd1WIQ2`7!}q7`*+tWLp*NiXh?G1 z#P3n)a2fABTfdi3?r1a4{StjEaPTW~;&#ddb82zn)D#hk>~K`dUjkuvboWK1g6QT; z-hkM$Y6T44?gs=`ra+0~rSYRA02i?>${#xN^O7LQS1jEh+GcVJrUxAerbNKi>R|H5 zTh4JH`bk7uit%{syRY%kf(%jgTtW6Cyl zqjVm#az51JENL|hg!TddMYZ*Yd*@P^fw0*`is$SnPeA%y!Py9}`22+sqX*CE5rzBr z#1FVg)qSt6L~xT+1;}uOvE;6ezG#ffGJTF#e|WE4nn)4gIy=-)W%1r4s5+VN+H@hM z$N)bJ5#z$+|D95C5ybI&kw$7)&g953-cPE&@lq=Io%*~K*0j3gqCC^RD517u;lh`O zoLAU|4u648fX>!2JyiI9d)^GjZq@*_MN14q1y4X~NdFx)%cKV~%o zz;WOy00LddnY~lb&4S>j%NKhi+{(7rH4O!!2qMoh=jRW6TY<3`qoE-shdG20|0oK3 z1>!yr9u%v>(|ZZVzKjUi7b)4KTOV(`QQHx{j~!5zd(~XvFcg8;b7YT+6cHYsLHqr7 zNHmJ`th>^klex(HNU)TafUO$y{>pL~5KBccdnuqcM}Cj$iW$ZhD=w3i18`1C^IM#~ zyPF2gn^a=A);_$!Y4AOB)J^WExgUo2NLW(B>Pr9#QW5)HAxLV?IlHWK$87$Eu zxP;NKfKNr%xD4qT=oa)}uzX5j*vcO%@Bbi&< z3iv{roQoM1n*BvaVYs^GX26D}m_3L0eoBeqE>l!uQysm*pfgRl!LY7K z?Dn-38&UWv`7{8wCXZ3aMau$PSgr4~+#O>8$XOSs*R{Y+M-G>Gfcz};W<05sYYgKl zF&FjTB$xs~fLBcv6Eqm%&CPJxJ*^P%P$;RJ`*Dd5GefLeUuwK8mM;%5 z9eAO@;7v|E>7^$qMXuj-sZ;zDA&>ZO^Fz7jrvt2FIJlDKzoLy9rb)J=J@c2ZY?c?SJn+ z1EHrVb!`4~3>~e9bx|h)osoB`_Xi8$89D&78dw^VjPv?9b62c-?;RNS zy3-H>7t{RRgps`pWH@mn^oM52VVUO{t2#K==CnhvU2;17*hSuX8l$?uk9d)zi?fhb zb8Zqg_D8~kgGlpS*8Dx$#<8fW{=G@jh!!)PyNW4p+iyaB!H~QFii|)d8(Hl?qofo3 zSqsqr{xKY+rVPVb0&Vt5(c^;J2E!>Vl=(#ob&m`}wg90YSb7=wfqA4Y0V2C}*cI!g$h0;sS7qtY7lte~k9%Vd%w+hO?@9xj-21IQ zqyu{}U8y^@qua!$zTKQ8%$TwjhZo6QOpT$wMU{5Dc{T-GZl-l&0MjLhhLZBo3+KS0 zA87(llnf_H66Kn^k3}060+ozQvYNjXLYNo2Ca_TUd9qqlx~ zq6dg}IfW|w{S4m=NMfnJ!?23wr`s#-y5-t&6iDt_*>kj4_gWwu8S=xM;B&4@-gNRQ zA00V*0NtIiTosgFtH@7NzC(y25E?hqxF6eTXBL1e}363N1>O&7jrx6e|8f4x+z(NM%zs zOGwn2U)mRsp9<=JYXLb$8Zd=0EZD$9-4eZ6z-Yt*Sg*HN-KMt}s_uaed^0eHW=b93 z;2x|tosu`pX-yO>#G-9)cOrf*I-Z09k1B3OrEV0Kf%H$mMM8&NF^X9NRWHBpA2_ zf=qcp?>KnvbDYpCbQBdPM06y~QQlB?6{1;f$=iWq_lmUn5*6{#3mEqH5QOg`#>DcH zbMG_7pSA9U58lnDkHMw7r#gt5A|;a;}Ef12KJHh#O5I@~-S68sHrf45D92x2N z9IkD69B)S?Rm%~_`4ifzCTT#j_lR)pZ=R}VsV2U$DP-B`7V2czimZT%MBNIp>P35~p!DVD82ML;G{RsfqC|v2TVtGaqu{vYx`Hj@w+BG1*R^ha0W zAlx23IqDG`9y?GJJc+U}F3PD@@AnZn9 z=>)$qdY0)e#aKXgE=+@!0_8DOm@g0F4h-i6{w0AaR@*t^7nlBbg^Bv5%oi5g+UBhh zRr6;Py7o_AlD;~c*W*~u(Q`W*d>>4oZ2z8&r$qbljD?=*H&?)CkI=B~adxP>*&TO1 z=CHWve2(<7*Fbdyp>8KYh6KULEpaCRm$J9vjDwC$XpuJ%o&0*mD5kl$7%KKkV)4G- zdp>)dtvACT>n%jfisPt?H&0AX54sIC>B@@gvvTh%GY+UEn0)}V76kX}cj@P1pIDcb zK7b=VW4)V2XkHJkvoJ=YCuHN_`k677eUeCqiHkUIM@t5CU=63dB{Ud;1Qy{DMHEwv z5JzqTgE88XqE*AGqm}F+f&JS$)9mPD6 zyWlu3xHrWQ(y1b&JNU_YcCL<=H9NFh`eCG+2%;+XoF?b21Z=kn`yNa|61ya+ZLH4` z0PuQU9LF7J)kcS{IVo++;tVC|ew#9D^i)Y%z|WF5IM%<0LkQ+@-t`c@NpI;`FW(#1 zs?sWRpLtocQZzMkD8T>o;LBzxkvaFM1}95&gUhTH+1Tz+$?s<&uOkr@>S12?;)NMT=?{iQ1WqEj6qVT9U9$%@_6(v@5;m_%1)||n*A(}E z-}vC{uTJ@JPhFOG^T@<4fpP7b(a26zr`R*tXYyEbRvMIYWKL2#uNFxF8!Odq9}#-` zKtIdMq<)G$x8!H^fSGjfV$T(tDPlix9`P7G{k*kFrGz6iXSirkWDnvd|7I}nWHZeM zc{G~ZWy2RA1`k8;>5CBlx0_TpLakou=J?cD`W$!Z90D0$0^!FZsD;B~VI>Y&@rk||eepwg&R^r`9=S2=_pZB`w%dKJCNOCGuJ~nVrpjO^-txOib}|(f@gvjtQdW(p z+DX%gNRN|P76T@Bo)K7=4qd0o*DyiN7gXfsS$EPkGxl~zwhCrqA;!35hq8gpO* z=pV&*spRvk>n(e~?_tmp2L?oTO)hE;1Nrej{%p` z{z2X6CSgeg%-RA`2Gp!4@bNUB@n?nR^$|~k*#sFO{w;Rdgp09Ly2SmE6c)0BJ+bqFExag=+nlZfiBm$Sx<~eO!f*~Ju!;~QqC@I$Hd2H3}Sq0_A zL4EPVXu<0X-Y#-#JI+n2v|Y-KvYKX%sJ;wUDPs^&WxPeSzWy|BHtZ(-v z>^$?~)W?E@1GWLMW_*f)%B=j&TEO;dDuNs@Q-D({j1F51(z0j4GHH@5OB)@CkP6%L zmLjhIFsO(`N%?(w{-W5tqxKVzu=hYrYWUAh?&PN!!9w>G$=|!?ipok#a21(cq zMQt);q3mkN_v0?%o}RxtUo2xV%;e@2!}JrC+XGNx^AoX4jvwU2hduuBCRGB9p^Lh; zcUz1o1@`1M)I9Qwqg9~m&d=V+&9w-zr&Os{VnZgaU= zYjX99FHbK^7v$c)6|kHs;oU-zTb;ArnXD8kvJHqw8x2*&DHwp&*SunBI;1`_MJIh? z>6BYoZNo-8Fo%4Bj)HmWjUPV2drHM7Dt6y?kh=2S)6~Y}yAjrTO%xJeC0;@Tzt8tb z@=GOh9$tN)Q^jDrERoY_|q1&14iAVdIo)LVpxA44WtEgu!%lgbJyl1lEt$>S@33*bkau>{=${GpJ-gMsE}|yJ7qP$eW2C z5Pps9?ucPaQRgxXtnn403-ZUar)MoymvhUY!JNPT*%nad{;iOg_cB9E+9*ni>8t-T z-IMjvyvphER_$v^MP0>KkUInVuOnaH;+L->eXtY)-qOYO*mKgwPy=q6yllx({wrmh z^Qy*I25=rR#3#`YDN{YKA^^ShR?Gd#imb=Re23V^1M8GcW?U~eckuMfK!Q?tX1H>_ttlAmG^Qd+crsY2 zmZ`pPffKO@zFBq`Brj$eryzq#J;45w6<6c3bg!S97Zc@THnTnys8tiL*688teK(W3 zO_|UeuX&byL-Snk@WECCJQ-3w`J_D<+XNDp98xJ_7M_wT_c_(oM=lHVC0Vk|+Xr9W-=-VHE$05TcK!lRFrx7da-2!kWyRynBYFaoHwx@8_l5(66G z+e5^OgOI8nrwX#FAXxLg<$_&os)H zzmdOeqKH$b9Uy?8+f#E1c<>annhs7&tvVE!Xu057 z5n|4_zo5F0!@IcU9~uh{yrXj|I?&5iF-(q4w9)CYailwrCORQ|KE&3f`4L??3?yp2)^;AEG6e)yRjWvk zr9$fD`*RGX#*H?);*9g{$%_x9s#DL4i6{)aKb2gZRDbP1nL3UGX!1a3> z9XJxbB14Bf7D23Et->lTM$5jxu?}9-$M`MhF_Gs~5CDcj?_T2vy*pK^-}T}YjPD)| zMEVfR{f6$A&^41It*%(ydmGpB3<9ATS$gKrW|d5no^pco(R@Nnx_v7+mul9KpK(b^ z(TiVrA8U!JNX5yS*Dkd<-vl&P5YVjrc$34VSSX@Ia<%xp(qtGEq5}yr@h}3gIB0`* zFDeF+Hu<=)P;_ttZmmVQ#ELC)E^tH!8UX;-TI>R71gEAU8n9RNI_*;{!-A2oUEk7| z{o&Z8SaGZ(ReK3?>kOMG{F&HnSE%1fe`aEh!q_5YrKw_-ozktp5IAL_nX>cmwWJW` zqJ4ar#42xb<|T6REjRXiLUE5J|8ycFAzSvdlTF|L?l1f=%I~XT2O^&ZBP1Q2U~?_8 zGRR0Vt=U_=ym&-MI1qZ27M<~K^_(hglIuOW4!3(*;3JTXmp`qY2QE%dK>PgCYHXwU zd)med)07WiPTdhh-P=a+y1;>MJp1;J&4@?%Yk(N$nq50CruxwRcLrb2ZZxL^b}+cwusS>}vm|9Q-=p7# zcuG~Lq@jw@@RH@J(8Y#5)27dr6NJ$^)7>*CdyX*n8Q>3IJo)dAmSNqUdIyp3kMkPY z{eMviY)p_h9?Zqm_qVS}j*sg0VtDX)41F%2wmmlgvZhV0cG*1Px-{3P>M)qVmlDp( zh$fq>Vv5uH{qg<6anVG*$++zlsuPxc9sBSq5k|Jt0_DYKugmw-d1w?-?2`AV90>Up z8Fd`&dXpR*WD^C_x2kP~Xa^{ecQ0`VjdTdIp7Rc$%+dz(sfB0Gy{1~|NLIRX58P3o z!~;mKo8hz}R5^HjSMxT6+f}fQWV4aNU*rv)uBXDh>A{tRy&pRCq!HHId;Q`uDmQF^ z%uRAE8+usj4R&A_zD{pFNhX?Zmd$~5Y-z5kxK?uDXBoBLlX(# z7TL`C0(Q(c1mn5&>f(&2peSR&cH{`}HbTtu0ahwz*P) z{wTgLmdQUVdwqN|%we;bQ>ouA93kX^D# zV4+Q<;KR@5?5O#dgMAlc?v0v_SEKnE>Wnf>F-owth}*Ad@9i1lu*->w4$R?HTxLM+ zn!ItNH>Zx}P9a(3R{f!HCV4U*1h-|?Llr=(@3?ghfMItmsDU9{fl$T7RIs|1QzyBytQw}r>J zsZqq}lDjG1-&5fIoZFEh!fRBPciR>jRZPM&P+0QrZmaJN6)yA)qg4M;c$ z*r_z{?<8-$7JjXpWz~BY%>Wc-v~D#}qTLTY&X_J;q+i?EQ0;bjdS$WlDG0C{ocwV_Vex-D3TQHY#(05E!+T8W#@ROc? zU;QK@MtTR?R&}L%I)&wVMEz+ncis2845#;!iM)9Y+887#3YDyRh)SGy zX5K6JSdcro+>;ICJ0odnG*%ht2&;lB+sezfT_wDeNbmku3p?8O@xL83@)XJHc|*fu zRzr$?L4StxaiqPr-rR?$y3xOOB+$J#dR;762i`!g7(VcKjviSo2idzDbaa2YG-8cO zb~`%_YWTHsI+mlod3##TW!6;`o9v`{&xs$-(>*p?LD&{Wju#Q0xaG=_r?z>9G7S75 zO)_-Lc0VG_z2^yjsHTh6a?@Ftyg`e7848}c0)CLMnX7Jmo8C6qjJPvjm&kk1I|^DJ ziZX{2(br*yqllK~_m29(b>m9NK4T7`G?-^cDFOy zIfs6x@~q z=X6g*yv7{2K4w>42;OD4sdn8y#IQM`pwKe7;piovUupFwWfiy7!ZLJ*r4y zB+O)ApIO#*45nJn&{*(}@Y}yHEd(!yI|XUy`Zm@~Mm}D>?xH`F$)nPE z`%(#6w*J=JYMUk*+@JJms9TP@cWQgBJ#_mHh#$pRFUS`KPcr=wN)CH^RckH?~mAuikXh3md$_PBD#$4L1vp19JFx@6TPK1>qrw$P+B zR8(#-r84`(x#O&DYGgq$y^0Q~|s*HHC9pzwgzGe0c)D?C*HezqFl^RNp5P`T!{~oLd?5>2#W+_!- z#RykwMH;@O)4LyBtcHhKKgL^iqqwLR2TWaNZmg{gE>l=o)l=BHRk`%mvetjd2TMSj zaelcbke%zEpdUo*#d~`YVd~*E88k=t=VF~_cz@Sk&4}NWoYXnk9rw|PD**2AXF*30 zp6^xFo%(9w29Z+AOr^qB@}shegbdERrAZf6two8aC$~(H!t*)eyK{+!mg{5u-%+W` zJ@?#I->csHiuO&8QGs+5F_$m~$O?x3SITB}L%ZZ<@}5fyKA5|!@z zytG!|#DV z1G>|31Zl1Kj9pHqj+wXa^~FW&e=)J|BKR3qA%yKyzxYf$eC~2vFh*g1`BhV4LT`zs z0|T?-+kspTJCFPM;{|AS#!NjPg0;< zO@h_3$m!zoq-ATrdofzxFfF=_Rru!HW#zDOQ|;w+?OY?z5&q|cm{X?xmZ4X~U13!M z3waMVE<-kmUkgsf9FX7VauygNZQJL!jFP0t4UC<>^+WpguVv?Aw=_W>yCY7?<2Ngd zenPils4!-7`Meyi+J~C;LP1(qU&$TN72k@942`)c@GkbETRSI=`D>If1FujM1ucv1GbY z6nQdnX^g#GNYZi6T(Ssau>f#KYXAM z0qi{+ZBMswuM3GF=7{16AhU|uaUs;_bS~L~_Y?~(S+>V{rwBC=<`Ie^dM=Qo^tqv< zt`ni-oDF@>v%0QjiOy3z`qhaZ^C3<3nG}X+rF|&@rcRIBTKU^4iv%A_7pryv}a6sJ=dAYVp(n*sN;yBZh|s&-uKube2+Hzw}BH9rwfP zqUj6%Cam12eI6Er z9%X-c)LFnP{oEHKMy`HY68#O6GyCIWmg&~mDZ`ZMFk;FKtGn>xi1p}!@LO-mWHw)v zk^oWv!<>Yk5);!#<;Zjai$o*ouRqgdm~lfE7r!ND#*HA&)n(H%kO_(f-BJoFZ%do_ zqJZ$k>Qaw%Y5Ci;nBmrhE~CR$!3d9+b(6SV;?(<7*98Uzfc$S4X7t?nIq$mo6D?*{ zXY(Y=V6eA}IX)-&y+n9HWx$JUGLgX@)tbWF5eQ`1%Gv$l*2w}Iej)*7j#EF3^ueB6 zH#X&rKv5#}_>3ulDWME({Jxz3)NCTnLyt%AbXi;RSnd{sQH7IVq1>%SJMLr-e@=_Z zSi|pu5$Roev!(xwy|<2vvg`hbVWg2RkuE_HP-&?FK|%=$=}@GkyJ08=X;6?JMY_9T zKoO*sZlq!8Zr*E%`_||2yMBMaYdvetnl*W@b9R2tK6~$Lnu*6KJF4?0$M7g#0&qIC z<#?bLpFF6}W6XcIIiwg5FW;ro1Jn^`^Zejl?FiA>Gv@d2uMH|ZSo7H&ett)NO^RAq z3%ALNI@$4+R&&n4lQZf&dP5sMwmf5DJ5@4zAC#Dm3m!~9r7;t68b6LRL96QG$zD_K zcp*pQY~rhxFzfo5v@3?0?*x3Z{LDn3in(HS^i-)s7+DNwx1^3adZM!3Q6Y@R%l5tV zmo{PK2+*o%^$TW^jPlnzuwNn)6|OqR0XRaOvcA|kU5rA4itTheg;uwc`6Ryu3p`q( z;#Vll?>HhHFqzNRsF>M_Epwms%}X}P_xkBcH-xZR#$q%w0=Y^+)isZ4@ z;v6>BdT-MN#1UaWY>AGx8run*-Dse`IWfUHfv4wXkv>|qV5SrqNmwmbaoTPtXi-ly zJoNB7?t~eE3o~h0wim_FLp5xRtX}CSuN^)d`(tPQY~ddQ4|YE~!s&|7RPM~td?=)x zrX%zaGcM;T=6XVb8?Qj>^9Jf4v%oz729}jay%?l|i<%6_;8l_DNmk{GZ#)$xD;@jh zAiq~C(BZr43pg9W4?758gbx6t|Eg`W)g5~TLxgS}H~oDg~f5KlVLFeZ&eRhB6bEJ|AU378l(@&^4mfzYD zY1fS15h!tt#07SP46ZQIaorWg^82o~*)1{o`Hn9ca2km4qO<%^9nDuL4>HP4@>*tl z;ORtQDhX`xjBf`Hy^un^ETCF+0%qbemI-QJ90hC#;!lzeZJy+5k-iUCnd4sUdTjfJ z@@XT)gg>A1G(x&`u)~rg!2a&){=1i3_n2-$)MU5 z6%aW1tk71qrh02PelL5`^o{IC{lPCU=`-nZXoI%}*N&fsoWxb?#T0j*JpOK2%;WOD zBUd1ibb#6AS3of-KbsM7o6rz-*zxAX_j}k*rB%T@@%~uYcrR~CnXIhpsS0yH>gd|KEp;uA6rI^_ZXERV8%_O-|ywNOw zUE#!V{fGQF0U!Z_#G zGEDBl)=buE0zk`wcvRN?(V(aYElEyQ{fn_E4H!TDd$E6roWTbM-|E|YS&wpPm-O;> zJgPO0quB5lu=QWX4(&>H8lC>I1hUgv-!XfYvmY4SB4Qe zknTCeI9t(`ZErodW)<(b{~1T2!op5_nbZS zT7Oft>vm?#lBiQ!+r0NhC%=|eFV@XPbs~V;fPH8AqW~eu{N+bRFVIyeCgz5y(s)G)ulF{{f@ImOnTf%k6vN9dzsc%t%l|zTk>@MES0& z-aNB|w=!n{@p^yti}*OGBF1BmlNu;WGaU*pYBa37;}T>gU;LY$1_O zexHye>olo-0r^$oSPqS^yNe%KZh|Rnl&DM6Qzv<@2 z@HSiOI=YQ?a87iT>)(7DjMQ&lGeLH4s!aXq4qSv zW1a{&f&0v*u~YPw;}sQ=@BT@uuK*vSL;--&9 zSYH^cJ~Ia_^X$md>ZIs%vCQ|`=_PJYhubpTO5rQ{voF7uGRdUkr-|MH3OJ~b-iQdC zW*iy-eBn1VS9j$SR=V^0GxHC&UlF4DMQ`%ghS|Ngyo}mL)^4~Y80)$Q-l_U^$m+K5 z*uxtf~rfIqv2&pd2=xi6SwFyX(sXYJMH-w+@Y8xpc-P9OX@`iK_d*0r?3_ z0w>xyy~a$%L}P79WkgsNLoqjQq1=e-5-EN4)iUvUMT@-qG-DDbJ**&&YQ3>#KC+_f z&H~qWSFaT1@*npV2M(W@+2u>W27Fkik;Lok5Vh}ZDSicj(90}UsX_X7AXKMDc zHFrr2CuP1FB+Eh>VSbAp@2uzA=82bPsuH?$DNjcIPG$OR856doEFBPXeb3c(g012F zMQOz2X_5iUXCOlR@^r9XNK1L3llm6n$<|5t;q~2OG{>&zd{6Iak3ET&eKxXalw2mu zGC5nBams7^y!YR^fDFoS|50xI}7`-`Q zqVOOVFCS5=;a$eqjq(w)^YSd?7k!tNHSUR(p?A~~5l2=T*(I9ITzrQ#r=aIiG{fp6 z*6je1hH^%!h;t=uOKC#XxjBC+G9|*APwy~hs)QB2_kW>VXZJ-+U~l_Scz_7Y-KeIe zVH^K8F}+UYHMr(gCa^~KY`4N`5s zYD(LYjyA8cM~diqeNU(Q7xRwFA8?E&$nV&NkXQBHn|5r^1&+9=+H@%;#D+!$__de- zfM8M`NKG)K%a>f?2w#x$0!$L0_O^PuuCEN=M`_P_`x&do8Revj^!!rW$4;fPgU zo`>N)Hg*=k-C7cR#@Y^DYV&g_>9jTLZGz%6qv3-S@p#i_bbmAAY;C&8b!D0-2gsu_ zpE&(AYX3zDQ-*1{i-DpnW6d*U`MH;esQGKTf{`^{^NN@dBdUA~8^!LUr(v8dY;e0F z4pjmm4CQzYQGBcpB%|HlLpk?GNSFZ+U_mXD+cdU(;b3Eg@mN?t2@&TsDi)A2zQ5!R zF+p;UolgJUL(T+(7+hO{^82$t<@BT{2yFT!OWUo->2lm2a&dSuf{(!h5N@3_j?MPK z-bx&Kz2L!F7zIPoV#Baw_|l~Eyt9I8_no;^B_Kh%fDp5n4IhhE+lD^kUF5r&%}n%} zVK!EA5Zpmt+}r-v?ICyVPZ2fS?%T#mE5EGHdNc~iz7A2Uu;_PCEjQi9!uL!-7xM=_ zd*eHz-QoFd*rY2iaYvV4Y_M>vb=-0#v2he$xgW>^G0VlZ_Z!r$vkkGR+kZ5A)5DVF zRz<8@?ZL0TTzhV<`6&l4EmRsuTy8U(l{iPF>FY}$uCs2h(<2sAP4Y^8;OpTX2UiX` z5je4cwZ2y+rYQ4}&Ar{Y4G>FCU5Y@U)A-;?oV&v+Nh|g44mA+iYIb;lx22t*$n|*K z-5`cAl_3jhm3=!T!&X`b+A~#qIxJsl_+voD3m^7%#1>}ETq*P zELEGy^`N(^Whiphk;dKiXX@jU5GRb(P(A_}Smc1dz)0LOn-bZX)?4~f)UzMRaqc+k z)NT(I&rjFz1=4#Sg!uB~)jUwj-5%wqTn<2YupcZn6(L{lYglMrG@$m?Ll2+eDWAqo z-%T)^Z>x;rG+FAf!p&am3tV#Y>K*CPMWyuIE${m=xbV4LRlA$D`E#=5PBo{4p4dQF z+i6))>n}c~*E}RBa|UG3zj7D`O%N^g*UoY8_U(-^gyo&pw<2ng1a|ioz6{zpObAnMp&j5#{Q? zKH19ZNJ4J|HEP=tyhxl@?LUxiMu?xde}ryIPch+nzMcdpUvJpKzqVix@?;?4zKJf% zjIga@lC(ggOK0U-w$~Q598!B?!t5MRA>3}nfa##6M335UTGmHQPq?~{<=Y9aiw|*6I_WK? z`Q#LCtNP>U(}ul9>LD zOGm%bn0Z^uae1r?8D=Unk8SoWZtY;mpKd2?lAR>`n$-8{tX~mK2r%*T2W%Oo|NpK^4jr7yKw1ku4l}S*3pQdU)Fsf`@idXjqiicCl4rna|Fao zIW=m)K(=2>l?kHMLV)(<5jjTW+iSj?)uP*eK6ydO9!7FZkiuR&b*%&E5b>$mSa^Eg zgh%EPR+Xpju2XkiyZiFb8|^^k{Yer3^a?%Rz71=N6{k4za+_YwMpe-pX@c&75iN&d zy)K}!i^-j=rP!6{R(i>@oRCP_9hQ5p95L@l>QC?xMFl)sOc@rN#d|fn)APEeaE2C< z#=4;zmlX5UU*BFgBAHD9nG`xU{RpZe`yvb<^QGYf0?&IuX%Uxq8qjE>vSvHA=T#oi z=)*PE7bFJejfwiyN~Zz8<6VH1i8XbPK}YPOwZ$h4FbgC`>JB0X00zl10&Kc_!Wu#5 z^em|Z5qhg~I|;%2btLKlpaRby+7WH(Y9m4U9HcJhb`G`Auuy67D~>!Pq}V;;j0Ij* zv_vpGwq0_LUc5zcdM5t{erBDb29BqXs0Pvcsa zDcI~-qu*}gJ?^>x>ubVuykgAnGlYaxr#Z%MfW^dEGRIvC-k4G;9#I9@IYv)>HyLDQ z4;sZ0$(53Z3=z@@YpIWf=+ItB*=Ekr6T|K`LQ@(FD46TzC5rz|5E=osk^l0oMhN?3G{T zVZMrYHy7@T*)P;{Ox)M$L4TI}ZU-;_JrLI(38gx+*?!vG^5I8wYs=G=nGWDQJ}TZ* zF%SOD2&IAjrHq(%4@h0!GB`Q1LX45bIT8zr<9e900W0AVErER+9aC{7D>Kgh@OSwLp>hoqW& z)iQbs*3$w`HJLAgZbI*AWLI#xijaHxe8;bIrXjS@6sDW-%5i1ihPuKg&4rw6sY)OM zZk!`O{egG($xwNh^*g@S+iR~ADbm9#sp@f|P%}&FYYF`rlBcbq$^DW%uI=}V+-Gl+ zyAIw}+Y23XaU06Xem4kvya6xi@wom@5ypyyRNiMslOJ2vSGmG7>rWQ#xpX)dQe z&l`GaX;gVc_fAjE4UyV=&ehdEBHlk>6MQFpT}B$KsYaF=s*Lergk45P67TNF7gxKy z@-v^|TcU1VaGs7LOx*!u0tp}wXN1=WX3Xradn?KFbug8et=vsA0EHA|xQh0>akzT$ zRuAC;`Z(5rIJKNa)RY`DHD4YDRUw)r@#Wr{|7n2&!bW{7%^cpAX%7mxf%FklI$07x zh$fK|jNfe`7=Q?kF9;#_5;IRbOieCK!p@53h7+S@0`C&EtJ^8q8p#ui2 zezuI1?_+ypZJT1Ubm#&&fT%Vj>Gw8lL0O=vs?THRjUsJE@lZ#bQ+j-3*mT2a5{*<4 zg>hWWy%b9?Y}CtC^iP|2$F{8!l@;HV=%Fii^OS2*@YnTwdF7gqmx$p~mt?p~yjXTV67oTY%3tZH$hMg9{eW!iWAxh6F>$f#;20 z-koL|JF!(&X2FGbxF~z;Ipk!~l15cj5~;BcO*}kPNI-wE-Lg84g_DdC%+}TpPoPvo z)KU28e1pIC7=7>1Zui1~p#jaHR#6B|yg*6*po+paW)WQfFm44Rf@|wE6378JAibbV zQ)eLoz=n$xu(1SgB>vj7SvZAY+sfhSuDXoK{UR6b2((`%L0Jr#l!8@^XU^4wyUKU` zKPv{v0*RUKbo6Qy_>qa#tahoo_}GAvN${7nn1VU|e)lIwtUMc&anfHE-KvEnit z$HxeVRO$LriVZLYJnW!+jxwDjyyhDFz~hlim66DPYWz{h*0<}>PE^YV#8>g_t)U?Ka8p44p=^ zzfR}wBRGIM`Z|vT&o)0!5Hjtxmc-XuZ~GRPYkujmApq&x93ph}7pd4@j_yZMgbuZh z_lH!D#e_V-yX3_W)^$6;O64Q06x+QE-$c0Z2*}PHtH?@FE+?{PB5s+-j_jEItbQjW z1G)cG4<`5qBu4a#7f%3Q=%3tT;I6M6O!9N`hRGwXFGZ!s6`VTyfU1d6IjVBBnjUaU z5Du1jx~_+IFP6RPMe#WVng(5zLj)VX0dunCnZ^ z>{}@;+UaYFkM)7-;g0QhzTq=5s-K=zV6lOkA2sU&@@ky>hdi5WZeCAS$6Uq7hx;R& zw%PMswgEvvG#h?i^caZ(HRd$cNgmf&=KA@aS`h)m{5<^~ZVQJ5@1 zZY1V4z^Rui>d*B?j@sus^+s{x04Z|s;qv5Kkq&$PWd;ND8bq?T6D#9KIZjR6*%G>= z8nFYCQL>vWl8;ujMOYWjWo1*BT;2dh8-CcVFJO9I-&J))ot|Z`_<=gCd8=IxR!dkt zDe>lZkNNQ)=&z6LtkRlTF#`#`}jW)T%H;?s}mn*ZsnL%}@U zu$&|3rfjWojDwYHNZgczm4Twznv`%MG-q@f9ShZG)kIeTxdc>F; zoBS4o)VrDHFt_;XvA{z~J4udYU5ZLHb}g!deI2eIJoOR2uvGQyZ4LHeKz)usSST*a z1Z5G0)u2>&!cyGZNZ{P83}aqVb{NRm-}`!a=wLRRRBfqz!-wDO9NI#B@+$Yg@m%}#hg5sZ}PStjW4g6NzJw$gNDvQ;Zug4jzlF7s7osv;>hVjKjquo?Qo}ZaF)V% z;Ye6jbGoaBoJR&%g+Lug!hInC<>P$wf?@;)3``vtLT}jk2^BpL`fxB=<@iJpVsij^ zNlCycNkkvXp)B;c%H(`YIaRY-%VE2)iI3M+J+e9xe1nqJiV zcvWVO*ko;wpN7OvxYW0dWoJv8giyjAJ*J!{YCbY5GxMzy@#uOf&-h-WPbSyXnz%^- zvv_r_78l{>1GfQZ#KAhP(t^f+Nk7$|y=b>Gc2ye^hddTytSJO?P{vtncPm<&Q6^pD|i z(9bCzRJ7bCJ1V(t+*;@H&0~hoSTGM~$pp8G!*dmrO%blE1JrRBtyc{hiipu`ROkvN zlbozR9@7CRUtFJS_PVgbA#K<{$%@gwP(-SmCP~VAxVo}_`A~u)B;j$W4T%%7GR;qEt;PLbBaTotk{vQ zX`ve|A0df$9of=fMqdvc|Lk|bKnQpEPVt;k^G-WkUUX&;hv|*Dv8Cm2$8i4{%V(^( z+XPq6u)p^Ctj+LOuB-$S#;_O)O<&=Msje-?ys+j z^(KK&J~$hVtpX*JO4JozhJEwW@r9mWqF?X5855CAwpIzF@G7!ME}+U@t5>7;c$vF0 zAL0k9Fd6A*2bd|KI>5IypiMp?YJFCV*cUVlAkK|+M=|JpmPI)ShA}0qJyMJ*ciW19 z+_G_DD;0-<7G$gDx~Gho*4kyxVj;B1RMjL{gQ->Pak>l(b{!oPtpBLW`R?}uLL#2K zbAd9P1wY(?Gv~gSJfFK57Jm zR6y`ey3JjY2pu90O{$Eb-twSFkco##*1KQB6FgjHa4B8{o1Te9 zq$gp2SJrxr(*vTt4|9o8wS%lb@3;5$4}Ax5yD`yxno}kES!ust zoT0X%;0vyeu)7oax`M8=496+F?G+^1b~CVO)r~|dRF$RsiE(j!%$j`lx3Hc;U=uLk z0LDaJmfv{6VU0zX&$sFDMeL|GG)6U}OlDp#Q_MIN(n6cj$l%kai^s zNg{SWQ$`#ZAtylgMUmZ6n`*t&lp$`@>1A4~iC_NeJb-LrGfM*ZHZ+7A{H;vnYP#3z zdLLo{uxPJ*wS4@x$MV|udASGzhDYDmBcxi~*6JACc2}c!CiKkUH4gCW&08jOOk}NY zYvfP4+h<3=lV!M>0a@R(BIP1`!N>449NS6QX;t>@BNj-|iiFU9Z4%s+NJNhb1gkYq zv|~UoS6%kmy~inJg-7A4{*X?RsQl$vTEgnR0Os{z3Lt_0<$eHz3dP!rn^dsFpnA`( zPwu03YrZ)1hVpEmD3HmP@15M~b2RB8qR2{0!3VOY)Srr1or?|u6@Hm25Bdg~+Xu$i}ILZ`gINIE5yUp#4|D)rbcTT%b})WPOV8cjx*?}Kx2@M3Nt zGi_x1T)&aNX?KYk8Ghbs)bg^zO$3UGd(X4W=t0|gJ4=tymRM<)Q5Rzbm z@hewh-Zf-XpGxHlh7Ee>&R5{FezxAaj-!;W><{{2Cp>5A3=6CVj*lHukOxb>qZaM* zs!Cc9;Axz_Vk&_vBYLbOgvXt^imx6a!j6J;(Ib_7*-1OP zO?FEYd#875xx!62dHlh6JQ)6=cN zTL&8v*9zGhl_D%C`<1SML1bC;Y^f9THWqBEDRFh~d6F*=yAK6GY&k?^#6r(lj^(>Y zIZa%l49ogq-k9yUp00}TRt}da=N?dJAyM+Lt_d0yV4_amjrHhD%}WsL*X}Vw$4l@5 z&{prhk$ncig=RT*{F2b|n2;SI>XxUS9Mr8aF&&F4>v+=SZDS7vdhH7BO3Lhpk%NZ- z6>?))I6N4esPY<}Z>zV9Pxrumvy$Ml<>BA=MqPh#3N~(&0)HEGSj^r*}ULLL<(jE380VV%Xubx9o3p2RM;{smKDOI&akD z8v+4JpK&)A)q;RL)RmGXfl@|39XKxjuzIQd%nxSq&0V*hd~Z9C{ul`$!#YnVUDNW2 zl1CfJ|9d~PxtJDyii^YWqG8El#~<&*H6^wWlEnSJsFpxO%SJV_Afd9#xFZ__30`57 z&$Reaib$g4Mb5E8>yr;>e=x+szD&3&irBuKeV3}i(rs?cX+%m;=SLn{m?=0Q=98ZjS?_)g z);`B_j}_m@cZl9y2%bd*d~8WCq`T zWBl{;7Kh>qjGX*DnET9lls9|4iel|X-&gbT9NCT|jLRX9gkNho*8p^~T-S(hpxE7*5L7 ztcWLfdiz_`Z#Zd4KPSL~_9MKHuJZRqYe8+}S5>hm;K9%(|%wciZBQBEILt{8?| z!j)Jiqlw{vL3dmTbm!tAz=v=J1q_q4#IyyxjQR}3Cfu^cijdB^M)zC$I1@F*>6JEvH7h&&gLsgSi;+D zAIk=;M#b;H!I8kc~MNc4ju@u`Zie?s4X zOh6Sefvnk?m*+x`3QRy7{3?YN*$-sTTmIvL>y5WNFZY1nDL1->I;mfe@2Xx!8bYF; z@36J`3p*VIW4D>Ff3Ipz-hD?lwuEBQWW@h`G|;xI)=cXM_Xs5;%Cm`b@&25;^4a2A zpb-N(-|0-2CqNMaM+GV%hZ>WN4MFffANGiz2vzpa1=tZ%kntF3_0u^_DdG>z%BlZ; zyj1m7+oODudxs`2lOJ&s1!H+{Rtk`ye_2xF4AxNbQ|P=b)$3UTj#=(`vCCoS`89)_1ukJa3kzV@2m>&2Pw~ zfy#%^Swn|QeW5159EuGQAwL>%iNeE=I7y!{Cts|O6+1aS<~u*3KvaI4(wNs&VHK;(OP6`DA-p znFVZIOfz`RtK+J!74NIr{+@TT*ioqd@H;NU2@wXq13+|IK5wO!tm$iU7oiSxL<7Zp zDVh_7Crm>}a}=tJXlPvVq;$r`jEc7atmS5b2K3Z36XrkfOK~1KxWZ%J-gMxPFqpM* zK4OY@tpPgS;NSIh-Mvi=b~esB)#A{gdchiFcMW1_<>PXmP^_FKq7 z(udtt^84Eb&wbwp^H2qj&76FuyrF_Gph1WM2G)nPu4#SSAH)u6y>$^8`W>M>2nEY`=y?}C5#E@A~YR+rDa6Qdz6ajkh=p^F%6|Km3^;PQ7+`t!B2Ke18@IYW=X zBl>9h1sb>X1{vR7!f?8dS%Q!acC_U}7c5jEkb%P$oh}0|Ktv!9F;O)RHsbxJNh%(4`TG`;SLA9%2RPEz3UtWjTMhqJO>~_6Ml6Qbb6J{+}!Qj|KkzcKaHz z*hA#|)qe=!)f)f)p|>L-p5W>nl;(?DKm5n!|BtsP2-)+5v~&C}ZvT(hK<^kpvN+fp zY<{2f|Kl}S6w$z#hPD{{zvSfqUY7|OqH%eftTp8yjQBs+a`~J8vFSnr|Hr0Fjr;$Z zHi3gMN8PtNY}?q&Rd#Go59Zsxg-~dXa>bt~Yn*G3cRzvi?vU}UUGkB(9D1r!rbA+p z;a~LXq96J`$aeoyHzuPYiHeikJqlzs+Qx{S(u;?;JQH9$((@>rFK%`8yMo;fM_5$p z!{zu(t9KXyi{mkSIoBSmDh%}C3&V!x{bMjR#UWnPB`9nZ7p>ucw^R$Zh~Df6AiMJp zHNX|1nCm4}d!Q?`2<9Wy@BM74t=f6sa{Qkm5ig*9z#L3?o|ggVpazV#CL;pf@uoU49KFfsV(aPdi%Y0+@QD89a;%-I1lL&aNuK=9QtX{vo<_!rH zk=??u@A+OQ@GSDKL_xLHDtMllgtziPbXgz;?f8XL@S2BXb5uaJ(!RU=#ejl)5I)@g zv-!pG9HQYQtj1+)X0>`P?||mS-D@2^ zk(Vm;;;u%2I;O)Az%iY?j-$F3(PTWf%qpy$YC;l-G`EJ-6EDzJ|A2D8+M>0hh zIo$}uKIMrKT&q#Wmkj*xO$mbiB7i%r5WEx#{cwT;L6v{(%$VdPI-L3B_WDOKzFqfp%;U?q`ABa`2@7ah}~D zVELXqY$S3pO?m!bC_|NY#?xpQ-6j8!O`w6e05CnY1GLHQiv^=Gdjq1>%9dZF#N#4C zG+1cx#(mjh{i#>YStma`*LuwM~+Y&;5D- zh`UK#hq#sD9ur4G^-C&iOG`_6S*uH9I(-waNkVjsS*m6p{Y!Tucl6Fmoi|=+6&uMt z_3%3ENgl>zP6&^4RD6np4@N@%?P-Y#3{)A9zxeHi-NS5N~? zUh_f4=D3YlSO0+4K<=Yay39mK@=qI5Zo!m8#Mr2sVfB<4QDZ8Ji$i|@cA!`lmI41TSek@F|TkL01Mg`$JpYSQk|NM%I0n z-kvNvVgFL8JM~OKqsG~0n$Pd%#osSxk62*0J_rSiTm6k9)x|WH=%Dts!-3^?^PR8( zPQAL(=sOxpVObmUSF=Pcp?FiDh?%$6>GpK|;y9 z&)!(m{VRZ%vo8X~kP$9pda1+Dz{Y5BnxzyG;zESbq;UP-_eCq7F-z~0;2Bs`Mmx(TI2 zd6uuca_8r5YyJ7oM4YG0tlmhV9RmKD@Q3DoS*ml?d`j7x3tbd^VCb_os5T0IMm_A; z8=BQ)mrqE@CkKuGR|8-08CXRr5nwqS%iaZ_e*>OF9fcsD94}?(8`afv<0GT^puW4Q zJ%k!rM@(ydvNIs)Edz*4xEwN-;ipL?DhqLIFz?ZaXa7MN?leGtLWVedFV(sivu5KP zMVeis!)+#h(+1@Mf0Y~FHITid`rwkpYHtebiwn;f?MaC*th?p<;2$0HNlVois!KT_N}-GU}_S3<_=;nBN(wz!L>g2*xp0>LZsfuqPH{iUFZSbLvqaE_{*KKOJ$5;~77dTgy#WuHkz3+{(x6pg<5{jXPpfC~Ux2VejF zk{Jj~ZPuT2+w~dtf#}JiLShP1V;+kbsPE-%3^&o_OE>W6Xx+gGlWLIPWAxilgoNF= zXV7*K7NIi&($_KFk)E9N#VhN}RvSV1UPX)BX#cvQ-(N9;p-9tz5R;X(`$awJrXFBJ zKHYCM1Yuwks+X8VuqZ~t97SQ*n~4@C*|0GqA_xKC(TmTI>7##ED>p_IXejr|W3^T7l#F}E}m_I}fV@TbUzB+i@PnS(UT zUw$W?zk#fit(ISgaJhideTquXl%jFPy?-YHyy}$%gt)3;OX;`O7BQ~_n2f*qIQ z8kC;t#j7WO3F9`uv62 zi}L+OgWoJAcn26m#Y*}Q<2Fgq{R7lWjcpllD=q!e>{ZUo60h(G)*lK>P2_<>aB4D{cIASY4>Y(lO^@vQ>Y zW*V;=yBVbac2?)Kdr||4Adyuk#czX~U?BtiFhJGy*int=F^hI^avD)q5H?g!moFvL zb#4$y`WM9gDKfWo0VW6X&4>IZ0^A{xA~g43*m?!VNiN?^i$Um}f@?xY%f_N3+-Tgy zzw6Z%v%Mc74EGOLzr)}78-C9_^>NJbTN1W}XErn7Zx3STe*5+xo_q$pr?-kD#rxj? z5UnuifzPBLk$i!gu1nKF;o0d?dirMX->-cg3!f#tK|WRH_cg@9usLfcQ7{u6hfPDC z@htiNA={Vp0QQCeWU<$g_x+8Se-h|wg~%vz8o)e_Ry~G8|2EX0pLN?K%$H6~kN$7U z^Y2i9yz(|81!x!RmR$B%OZX3+&gaaHz(R5Zv)niS#)dyXgT^8d^W1p)%5Q}E8wU`d z$uR>!sK-G7=Ql3=hx+~V*2D;q;JC^?Q^DUP`2V=Z|494~#s5d*D<%1#6kjRM{|v>I z-S~^4DAuWV93qPMe8Joy{KM7FjZO^aS?Cf}Yt@j1VegH4oE>dK1UdNwIV!8i2w{oYW&#YiGqa`D2Q>eaaHTV;fX z<7rJxZTb5(iQHCs#VzhXYw*71jcmT(n@f%rhA&8v)f=eYYCtbpuR9oGm!FVJ=IzKk zX?!2}J`WaQ5SnaUTdQoo7#Xy@jhe2ivRiLHX2iBy*OfH5Z!mvgcEQfCaqFqRec-lKJjr z>44*)nV>G`D__#6wThK!iQ(OrDjdfH61DlZIAM70?)pPBgP7c=)jb-%0mrqWX`yWW zur+rxxn)~f;mJc4P5sL2=vhb4r8>To30tiO?w00{DlcZg5Z&8Q;JAqmVVTO~=%lhXTsv}`-sEHdSmEmGev-iS zn4}al_bVu5xZGjnvHZv=7^0023yo|*dLBKX?UCwnm{9!168q%%!}L5U3d87WYWzyD z|JK7~Y@kgCMa)nZ58j-p?0x*%Jd!@P1L!>~hT8wY?Lg#3{%V6WpXe1~F}fXf46i#Ia|rCM)jU!nM?G#JWbQbECRSItr=b}JDDq-Z zA%FDDnmPROO>Q?9o8z`KH%pXQ=?aG$A8OL^x*G*+rbyS{$op9xATO~Kwdy}g%u0gM z${ODL!jj2hb1jSGz0z3$gK@kYw^%0Es2LRt{!2BP_?=)Ml6ajVlq`>WFJz&>$!n@VtM#c<+gZq2RPsdC=|c_H-raUQvfyhL zNdrHM@mgIvcJ3E^OmzSSzwYRImi$W}UbcdYOzCPIDfRHH&9v@c;GXv(ptXv6UgQ7Z z#$GvCJ>2b=kMc_ANOWJS$hbKul{EVVdpng>6__=NS{m7s2zwlI60nQ@(G1@%+vB!P!X4T9Td z1T2$XxA0R#Pu-@_hW4)+sOV-+evn-v+77-?OKEpp;{)?kYT%Wrjk3Ms{*@*oOw3I+ zfZE?yC|&`PqYI(wRRIURZ&MqM*K7BfNEq$h4Xis3N=tQo>t)b4?9;TXCt5CGE#^WG zLY=x{I&eCxrfq>mYMuWGgK(WQJm$e_L#`cr&xD(B+lqC*BorbV4=EDz)6puo>_UI* zS9#Fps=uRI{VPoi+q1e~>;V?x{ysAqS#X1rl#^d+V%OgwdJ!FfT6qs6o?VIy_#0}; zhdRp>gG`gzM%45na_o7jJhOM^AlCMH>PrY(`F&&0o#nLYP$}75x6WFvi z;D{cNA7+Jp$SAWE3cDiFZ(^$`3ZUdAzzh)Sg?D#Z;ggKZsaj+}eVL7K}# z%NYU(<(&yO8Bv&Y3z~JCR+59Cm92NJKnn%&aL8(qqx%AE>Bel(oufgFOnCci_vyc^`Z4k)PUR zIwc-_mz39Y&38q?%;=nOF$t+Po(pEITYdL8j~T86%wB2OMup;1CgOnhnxv*tiXk^y zCPlewM?YnI%{vk9Y{w9kjz1mGMGBy>s^Y1~$SoVQIbz@a=wg>5WO+Kj6HdBSzx&>e z5R(*Pudsf0d!1Fo-;#!&JrcvpU%9*TA&d~n2zrfoHrwca^oiq>6llmvIZ_I5pNq9( zhu`>zhT|BI8tSvQWI0+{G*CG-O7#9jnT6N)1H;M<=ygIYO&7rJ+nBsv-9d}1pdn;b zB)N_Emwi+$ArSdj9+NS+lyur`kS4#U{n+cs4mZcP{LN(3d)9O94~bIOqw{^*5~n>6 z8xJBvsVh#u8J4~3@I7#i)`IS=PRW-%Pil<1; zQmFF7)^^Vhx_HqzdfA*C-R!}cM?OnyXi6EJXQym?DA8-VG53vFq*Edx-cT@+4+<-l zgV0}MZ!ZxsNAq2_+{?g)_bsAzj?Ypu$<%L}>*GaPi-@5Wn-f{x8s<*Ou~G;{c^sEE!aa&6_y{rot(A9-fXQBg5jb5x+SKfZaa zfw8SII*s)y_>9G4QBN8V`})RO=fJ0Jyn3~S6LG<%uH%`hny2dPVv24x2G0*mfV?_s zS5KAmKz4$j?ZO~2hb1Nv*ICHu9nc6=qJP}>1xaoeFLFL$I&FsOUHtsb(%_ckh! zc1f+V9x8f;!`3Uj)T%{#rw-kWIx*Zs;^ik!6ak~PKmIy$aq|j*cCOq^nKLN4(C-`>e=dgmYf_CpcNP`r(A0)Y_3xYu;x$ zB61ce)xvo3N;Z{CX$w8{!EKTg9KJD4#acVEWD5kM-YV8gn3$+?aBHAVlwEb?Ve9>yEqtwPSC>pek!=_$sM=s}vdk5aB4>(o<~F z;fn7fb`4Ai^HV6vEVXqMgOgU>DRAmG&R{l%F9}1%Z-?E1&Bl}I_1E#y0zfsv z5g1lE=o^CmH!N^V8(<|S^Su8RdIYba9B1}S-)THFo7F?vOE(Sc!D0{aJdO&&CXFt= zn>XR)WOW2znh>TgWd~S0xr#RMsmgJ_E8*Uk1C4?0#UAs+5>uC3C#O|AVS?31wK1g0 z`L%PhN8L4*-k&uo_5MG?-aH=4wrw0|7`v!!k>yTBS<+?=qmoK7mJ+g*HM{KV7%EDr zJ0k0pMA@>hV@M*|vSuBTof-RJjQO3{sQZ1M_r1TL-#^1ObIo}j=do|+F=aa^tJc|# zZdPOF+Bx=S5SxXS-1?`~EZl}d`6fk!vp$+ljQkWe)#GcdpPg@Ia5&r`aj$xoH(Wg33E`+K<&2f78PR!}@}M9uL4LkI}J~*ljikz!;dCUv z>Dk?U(<7jO`lq!_{;`a|;MFsWC5{_pz-r!nA$V+QeKZKX&<+{>jG=rNhv(^(i)sup zA!WjPMXXt$AZ{kT54vM4RLrpbNGKluZk@WSqzuC-wpKS&lGtTSTibU!XP7f)sBL98 z!)v6bvq$j_`I_ajSK0MzqEQ#E6lf4KEsOkH&)upEl8WS62eRi@1mP+WD+O~PmC1OlUXa$ZB~=hg_+ht(%%u$;c`_@A&}yAPX8c0MDbNr$)Ok?9_~x`c{=v?k&DB$$9x|cJpV9 zY!lJY$O4MSH0u~=!|J0GUrd)>;j8qnR!hBuN$lRy`=Wn~!BhG(q~_d`W@mEK3q1qL zrrD8CKQ()+pWr==X$NWpczu7rxqCwt&XQcPSp1|=+n3!C`FWX_G&vam*+=yQ!+#cg z{^%%{ZpeAx+FJ4c6Odqz!gd;LWL(rJzrJ)lY`|E-J`B$+$RpV+u6m%RoWCtQ>)dtk zBVXs-0`1Ih=@uG-luGC z*Y2P15wit`Y~B9yK&i-1Nu9K=bL^HSi8(`Zyw0bl-V=H}c zND5Jlcg+h|uYWXxTm73oMN}^zLwrd!MkQ4*#LdkBue~j2Z>G6VZh7C$T)&PQe=1Ei z=!s6K{n^l;2ph03ng=wmJR56EpZ!NYDWT_FZ%<0^R2apIvuw9dO9!9?x32b-L=bdK zELz80-uripBgMYb(aT7>4$G(RQzOkS=y<~@^X5Iae2b(t5IpcPx^DyxTf${-L0Zdr zHBD&+IhWEo#|VEQoR0i}tBzb8_`VZHbe1%}DFcItMu#I+w;Rniu-zhXdC40%svc&cXlq*ZM=NkrCzESOVY-tgv_yn0mzW?Fx$ zBOHtCw4_Hpg25l0+wMp#JRlJo9kM7pl%1DfU0ssZe1~L@-inQBDSM%_g)t*N%x>=e z%K=lndXHJDj(z7_RaXAGw1r>VOgTOA#_R@sc-+S$%|QJG_E*u7xt*QyNxnd?wdAB+ zj0jrh&iIpm5AdM=jCFDX-|wCq#D4Y-gR00w_DdF#?=k~DHR__L-bNpkY_yO?){^@# zdf&NKW%Ge&dq;R@zp)mvO2@)2aoBxO=D9`wR@&y*hEhi#S+~EYZ~H4`qhW$yzI>Sn zY7zcYl#dY;{$^-D&jaq&Pg%e|%hga?d)8B5g%2=fixz`{QLRGdr1qqMH^Y8|GB#nn zzkGl-avdyHHL4qn3M@-hV>-Xxr&8vJkG?bB0PT@-$Q0d{} z!K8DRx(wgsmeY6Ht8|qc5gTw`s)qQ;fdTHh{bjP%^a+gdSB>Xa&Gi|NwrE4%AUhzi zu65r7zCFA4cJSpF1bs%c`uYRE*E}PdD5bdsI%3HffxL|C)*)pz z3QzwfV~CpHAIY}0noY`=z7?`=)FFD9dmto8A(owIaoyrQd3LWPrg9@D8OimkVl{wUHwu$0i9IMtmV7$FOb0P5#iZk+sNH z)A<%%nKqQ+u z*^qnQ#m`7eDa*+kmv1IlfCirbW68yYK^gLGdRo?zKO{kxgzyRJZZ>Uif4YxEj}&!f z2j6)B>lBpaetplz^~xb6;)L6`@8{Qq%ZBw&NomB6@ADjb;Qh1xl}GV@x~27*3adTM zW{n5c6;RjEd>O=QXXEb+YfX~K<46P^E#zWiJ0+_&Sgj}vWZHN6>& zKr4D_S5_+$C;v4?whD;wEVbs%yMGjT5A3QsbxBv6je}3Jq}*gp`y0=*1_T<)&JLD3 z`*@GdX?5@(L4rQ7m-*H0z?-eBq2%@)In67XfhDz&F(kNf{OG$WBR52{6<76ymT|1h zTLe8Ws%(eNv*SCRUcOeJs;!DSYPbSBw8U{|Lh~etl(cdveU3ycnnH3 zk>mcSw+Ib03ETk*s-!kYk_DbhsJwh{<;B}p6HJKywgren{0<2-a~C&>@=0PsqT6q~ zxd)fkuvG(}R$1kz6xKTT8a{T5OhCb7$Z2##jr?HDWOOY>P6F)Mu3BYUaQ+5aUBUhEb;2{LFK<{oQt zK(wFmlj>pcUjm-DYiHPXJix)+fPis*iC~<%?~!`!r-W`tnwp-N*i&bvvq%=KuR1)Du~f%6yjD&ZO|7au3|z& zy6VK-;JQ>#Q+EAu_n9@mJxj>;*8{> zq1De{L{+RNA4erQ4HSpX{|xWtJC`9Y9jz2>t`_jAx0ujan4}c#LNES z-WAG>!c$`Sa8VNj??zNIf*xxTv|6AmUFbY$(QKqu2P?yS7tyL632dp~4{a5h@(3(i z4OaVUe_!I+aiZU`Vs-(Jt~aEX)*=KY-6k{nRNDVk$IgFaW9LEIa+KVg>s=@X6)LN}pbMf^X(qkoNrp46#a< zkV?7A*AD2a zR~)lAjD~G1{YUrJKQ^r+2WsDNZ0~aUy_u%u5YShbU*6cSw19gj?D@u{#!l(qklXK}m)V(hA^{hMvqek{o?#CHQ9CV@ z+FptY3-!x~l2TxCuO3RR89k$7w`9C&HW}xTDFb}S?Rbl&%{P`$@4pfZb^5@{gY?s= zgU!BBaj81;Q7V+cRp~x5wBNaH-dMAao`BsE17U;uO82`$D=8rFq*P2TgN+)pH%zO~ znT6xQv9mioVy8VOU+X0C+#0pjbR&Z1vrbpp_J3>AdTR!}-C~m1+V3(C`W;Wlr}x1d zJ|zJsi%*lI2Ve5eGsT2uh&}iQDxRuU6GlfwyT-D;liQO_%jeF`26XaBip3bHox+2X zr9nsaN8vRMz8)sFtQJfI3jXjmsD-{U+Tx$XDo83Kf;L|JiAqP)BF5c4yjOiT`C(s`PEPwYAx=8(?U0%?K9^Hnp%a>~oU z*(a!lUP(~pEmWtOs!t$k5CmtPI4kKJKr<9DB?cFe*7r81s;E~ zqKCC+V~W3&XLwFFu?kJ!EVx+HUvymF!vetPUIyA>?`7Tp&!)cLn|bCB)j{wNqrQiH z|E?z_aIiS9O~=vM+4<>JJ@#E0J9-QAa^P$2)64oEL^X+V)&8O63XH zUvy+HKP5EpOIZuOo1Sb;3=+#l!}MgrqL~-fO}~DJ>)|aHf+SGZHO_-O`ZWR>>$IwZ zia>*f&U^p3W_`>_g%Y2uJ!twJl0O5S=fP;ZC$h~I=-*AE)!E3ojA%BobVNyd{M6hr z*g2WV%C3s7OgoTP~YSe@d94JX8xXex4 z>uO@(AKuj{SJ&nzE z>f&$7pKd~%tYzYE+Z+P<@d^$`wo7_^;8&wZo|xxz!1J-C>A#AU&x^euKlSEjN7ZGE z)*rc``WGtR?5F8$HALo-*2~U0>^}F2GQ*P)u_>2ws#_4Ui&swHdicP zDLZ-9k4-S5s&?A!oQ}imTlBnyIKx+Z18F@0p1(c^*ov6AhakkOd>fyf@~95lDsJjo zi#CG?B^S5_|FXj>oHCcXJN}x9H=$|sqXui%??Vq?0-MBkJ;d4sE`w7nz#qG!b@uLA z4{BG^x5K06&d#~=?=u@sOiTm|n!8zPb{_p1p1`$VA_D=j4Vs5dPW0W!uB=nE-HpLo z!XLlb^}G;l;J&V|d%$(i%y%$12y*w*H2!78gs$14@9IBPX>o`C>R)fDH!q<}OG^{` z;`w(S1|&PckV%&QQQCq70C`=#R^9wGRPur4OOP4YXiqL?1y9axeA5??Ku_O%Rqf&H z`@X}XEs2A(m|dj$lWZG@0t=xp||RPmgXQ#|u)=-pG8zxwZ{-?e>~YB}-eh%^IY ze;5$<(k$mJtsI6t0j~VNuF&Er^d)ek)7ugIr$=@T`{(A=1@jEP zjbS+DSd6N9+_a=%Gu$KuDLaSJU*dp-4OHa4C` zCOV)PuBfWM1BFXRH@A*0=KGENFB)(3i7L7h7CdIVMn-H^hmQxGH}*Y1quUF>nDWNq zUErh$co>3y@&XeR^Z^~rwj~@K2%uTf1g--NON=Z$5mPAapweCpBfRwbGiJYYKQn|uj3m|r z_VJ#ecAQqjfA$Q}_WUy*TD$Nw2Y_lrAJI6PBXapN;NicTjvjvmnozvgr>7IJ6+Xlo z__q8UVU|Ocuh!Z2=W|$Y+uh*zPhle#rh37O7+738)26KL2PF)WaCeSJ;k{Z>h>Ibm6LU8(^W%@yHs zOdA{>n(Hh&EmmHtKNQQfhXG z7d+cN&?dG5w)~2CS12UUJ3TUgx7!eZ2LO5Jd?2lJ zr-K7rw03-3a~21bSS#B>(+eFuh%d>X{d{GKID{W|7G78=FnQy|*eozAB83Szjcd$q zNVa0A)?g0{3v1JZvt3H)?h+P^#jIzUReC->uccL8+2FvUK%;{7d9dsK!G5rZE_K_vGOan}1>BL6$A>Fgc#9m%@Oemy~LvoO& z{FV^7GW^>sy#s=`Q_1s9O%Wd3i0;zE`Pz?ZkG=;~!!>PPJzbiBngBq0_2Ua^Fm@gS zFuqKElS;d>1Oh)p)`kp-#&Tf4$+JE#nUSi##iodR@j-T4nMJjtr293$0?1C*7PBbf z)0lCl;_^Jrbx9QxwvqW4AHj54i*;izU6|6WKPnFK=R1d?**s159>NdVOV8RkL?NfZAieMU{cU zTx|OAfQe~}o#7N3tufU^N7BEEZfjd2X$q?o-k?lPllIH)omei-$S&%ttW-`v_eGUP zHcm+Z>>STzJ=4KtL&#}KfRylLJ?lZ2YUuEs{Vm$%BjIF2j{o}fY!ELRM)uWe{F0m! z%Er|3sWh+PWKqq|P+pLNBtmP?Di?g>tbC~#QPQzVTO)gz6|9y)$zu^iWTejH6PzS?)A(J*pAT}8?_cfLZ%J$Y+zab zTKNmlK{I2g0a`EAFoXB$0L3%D&{543qg=JR;B3=F+%l_BM$DE8uMV*>A&;YE zK2$MavDo1&Coxmi@!mUgr-$<&ZaAD+*bx;t==o%3hEN(Ez3vd{d5W#vW>%4*M5#F= zozr^4*V8j0^vJHBD+F4kDKH`G`86rPndn*IO}*mE7ScEe9a6Nu5VHAfQ%*?y6|I%7 zhlaMB3Zb2wo(7C;_UK%a|xdE&ul~V@=EleDt=By?RcFQ_a*wo$jLbtoD?eh`T9=RbGS>Dlv1HsiUqR+ z?dhf=n{MXrgl(n)u*^wwxPVbQw@K=}?QOgh+@t_mZF_UTh&#bQs?*i` zmaoCt?CS!66Kh=Cr-LF13W}8O>;xZSbn7HI`c&&r&)uSzc_fullX~Fb!BfL3HEu3C zbZElN{QO5k-k9(jL8atl9$hHp1AU1laag0cFJ!>0jQ6fq56oI?E#5X4rK))5@7O6l zc4Cd|O`D>L^%#JUDd42&7a^@7Xa^u?0pV}KDuI@-yh0Q5qbH*!p~LZ)t7uAQpwTUN zHfv}TsTOzvTNBK|Q^wq`06=6{WSv9j31LS=)O4#Ppub`U5T_}=M0w&T-t zqHY9tacym^!NkmgxEl*^Doszsr9TYn$?Vj2MtQi(ioG$krE_%)9Z4r0pVp|`Thf$> zyl3i5t7nVd0umBvEhqX&$Q?Ry_RJeF=AwwT8i&FdbdXVO0WjcC)uup!-P8Yw0icoU z1Hfj6vJMa~n_)uu7I+0l)%Vp78f&O5t!U5>kJ~v(h9PhsEr_K32yV}N&&@PgoO5&K zm2dibqLV5x>1GGm?$e*KfK@?5k;*0FN$}>)p~n!GuYw69x5vL!D{wjQ=!^L@FLB7=W&h4~KpRuY);)oLi8y%7k?cG5Hb|Q@xLDZ8A z)nAsHZE7{dkc+D<(!kH94m&Pk2RL9m8k4=+pi+q{$r#E|-!?HZ&22DA9{lpf`d!GA z*V|U7W@ltAbb6Up79bAol%|Q(=2azS-N$XSYLUxJb%7T2r7H2RhHJ6 zBck_qw40nU8Vw-@ZDCKI2HAx+_@Ux5m3}6k=iOah3)=$$QBR{2M>5)npE3}p*pxfm znR;%!s2Dyv#g?U$A>02xZ{(XQ-|M3EF4nx=BbfGqfGNh*?y5O$yK8gOP)#$LFGLJt zp=kfD9u3N<@t4XpRMYhTQPp>n`ACcl=+L0gKT%Fqq+e(#?vAR98v%#%E&x)1~dQWnf+f5#^rXdVBF&|n!+_VMvD)^_|> zQ~LClb|itAr(6``)V;@y?Jn|W$bF+coe$dyUEaGB z{j@Kg%nxR`fTSi)R>30#1sLQ9ph?H@f^=5)!Qp)E_5hS_9sNL*y8KJ+sn_3>91yQK zc@*VN?Yp;E-QVXa{(b4J036#V`Wbn}?;I_{=|}*uOpqULemvPIQv&tr{mln?OV|9b z>4`uRd5LE5&qSNf(%40Y#xAP~Z>l?CUZ4gmJK)$D4BGrERxWkF-G0uvu+Uq1f;Xgh zg}g;hwuDi20(}ZJi2BG7vJS9pk}x<|aY>o6MOv`xY+x!opM6PN^d7sqk>OnRGiR)W zC=6{xA0M9{>!%fK_vf_Vg(S)uNCGI3Mt}uCEvw#5oB$(UU^p#>>l zxoYNDT=lyHQ6PPL$Cv9Ar1G@RpcCP28~mFkEJW3-wPwXeXG>P_{TZ&4Zx{QP=mv6$ zgfWu5Ci_Z?S;}T@2*!4ui$hV1nQ`H>5AGb$9)!(CknoUKJ~dly8o6Al<%Kj`dqvV0 z=IwcVjRiCUkA?*2V{=tk6GH6GUZ|L8G_qz1?~bsTfqdW^d;(Xt!dpf_cW`>SCA^0O zK>Iy!q8nJfk6R+(u90W^Ad~D5c=slr0o)~|e_luIJA!^;%!jXqfp4<}llRbvI3YU# z*icD>@oGuFrXQ*EE;i{bf}Sz#g30y?*+%@U3-$HAc)Y8OOhrR5Q72GAvNp-N7HdH& zDJ@oBBhC;Cdwb;rJc#B~8}zQdVc&r@7Ccy3_gZ7>7L1q?!c1aeWZDhHJ%DGQxSy^m z@JI4^8ZAi7+jU-cX9+knj^VX8$$#OYZPC#zaZ?%QB>V%wM`f@CLv2$;RCM%-#j5K! zngf9|(!$Vq6Fy-oG^3g4SU(kH+6vcov8wWzWtI8e$ZaybHMu@ zwwoR8Y?qJ}p5#$Z`G~RF-6$c7rUicd5RV;z$~XIg6>kbuJSiSU3*Z0WpLUyIBtcyB&wViGf*mrklv;(Q<7BY)*qGRLeAg|S@ zVG&+khj6k)ci7fiZ7qhAV-KITkXwwXq>pFFfOOhW%xw%>R`v8f{TNQpJxXggE>Hy?jp~la^u3D%p(; z8wDnGs}UL)z|@`Osm|r9_##PsscTuVscEo=c^zXAg>XL5Cs+bCbhJEjeIOU#HEd;N zRiQj>&)_N?3*?jDKR@`UKdPtoIthGs1IbL11PI>n*_h)x4d0t(#&*Bs#_V*+lI6qa zfhnuzJzD+D%F4<)UiiK26beHHf*$YA?0)@scv}Nttf2BQihzCG!k_#)2(h#OE<0Nd z(IyOGzSCS0A?IuQqR_rYXxmy7@To3ZhiM*wuM-wdFYtI8EI8_^O)u}f9&Kmh&Kw}T zp={{v+?+GO)c!~)A+jSoLTmDt8Ui!6h~Cjc$d93g%+nzJDoafzY|rs>V3sNDq;V~g zg#D7ErQfj@{dAX-t>BNAZItXzl4r+*v;2>R+yObi_aK#=>E#`5rxh_?w_Ava0&$4q zymjLpHK(Ubm!Z)qXn&f2)ay9E1Z6cb&v!R)@0>nAbZNf30l|RkGH;kB8s=`EF}!W^ zwy({9K{@9a5KHx}>g_R{1Qr*)7ikMa)rKamVREk@VDVq>=U&yZkwJb8o6X&tyH9Ml z0dul`zWy0x%y2&~;6#Zj@y3A>4vyxh7%?V=Z-^sDcdg<-3gMOml#uJbM6=NH;*t6g zwmoUUzH|JnZwf&tPlK-HfN^YMG2#X&T73v2Nrve6GY?&L)G&L+upQ605Z9@AwRvZ$ zVH?cbaB*+mhtX!=sLaeL={crrGq?Osw>N({ghALCGtO|oeKJ}VDeQ%+SodW(I5d7> zNSfn{&%Oy-o%Bp8@Zh331idD8ZRmkRG{1Cp0Yab2F8Umt^kgA*c68TfM;XmRCC2Wn zH0W5!P@r-qPx_BF^%1GAtwFK%6x>Lf(dlClogu~flo6jJhR2z<7C^WlxZ`Y}Z zjPpU3AJddelwEBMlpOFVyf5lO^QNBZ>?+&n+qB5>iK6T_X0v>CmasZ^* zlb46b8Hcit2(w}ehoyk3@gmF-eVX@tu*+3O-v^`vb^xfZ1j0`GIt|pXKyHqXUp_{= zm;uV*nh#Yk01%Z9Z4FrPh5K6`Mh6+>4NS->#?`f{XiiC=dp@|NM~aduI3Ecd3T=)` z$lAW$8#1&6BTT$g^oB7E48UT}x~0EdHMbFSi&1lH*P5A~^^BHUEUo)`N$!242}AUi zEU$2+8bCz|VhcQ_tIw{RHq1fH4|qAht{Tt zGK1~{l&|$+pXAzz==}n^B;rjZ`vRdYIOWE{2|@fO-UVbZ-dW74?kv}b?x`l~Q+O;f z(u{-x=H@e_QPzpwUC!xNR`POkO%=_0%1;7lL^=ql!bD5@LivcNkirD$+93aB1^z$? zNd_5;=F12G*K^QVG@4gSCo{w>EHNry1#(Em7jS=nt6H&$Bq_rXjRJ}q%nQ!F-$(S; zFT=0~UIt1RbS!Ic={-Z21v_-rjz079@lq!K9W*B5E_SWcd4QGw#uuV2^Z(7SvI=`EF?o163DoAIhf0Cl5j z?jH<((G_?guV8RPms_(a8<+sFlFTPJEOO{V_SkR>2zd;=4hc(j5mra)@%pjQ%)=EB zRU{RU-;<|34h@t)1Fb$lM-Or}m*}B)f1{@G5K>B~( zt(4UrTIElReJL*YGp?enU8xOo+cxy)&mnhMdU~@9(sv#`GRGvBl}E@BW2AJPWQpsQ zkrLKXq2YTBIDlUi##w0?p9kBw&DXuRmj)0N1#Q@#qa8?irV1|R*N(|J=%5RM#@X8W z*u5PVcA-~YZdCe{6cC+XQWaZd=43HCndR(*W)qz+r-S^Bw6v?Ur+k=b1}2YDI- zM?deD?OVla1AlN}uL;rQxCBW@C5kk(BVY6&8d)B|*xsXusGY}EZKj~%|Ckp7Tfn^_hch2?d&Z^lA z@i#3qb7vsn*_NG|*mDH^U-!@`U*8}76FB_a)LJXLRp!t_rsYGg{dK0D)mihWuiQ9Ii446yBw>FHZ~0X(-b>1@g>_ZxXRI5-As zM=>un5olQRnceha2*L!?9i0ZsZ9~loYSXmI6*a}q~N5K;`!R#cYHg%j2J{{bo6aG`rj2uV+iuAx}8vphz{)l z9+v%m+?aNe1D3bVkhlkBlR^+@!IOKYm!x4OyjjXfOT$K14o;;;R!OT6aK6zffLv1> zvhIC~vAb0L7?g!TV06G2yh5i+Xb9p2j`d30S%*jP0RvmjgR}zrfOhnAYJw-T+7MLz6q2~lXK{IzNzXUv{-1ah(ROJ3!_56L;G9DFh$5DEg6IOe^k-fdJ{w?UMv8OaWsv6B`F7mnjN`+tl5U zv;;|#BqcPxv-3fTj$1)jooPGW2U#0|*fH9L{xR^7W|#C%+Cx|Y10ChK>Q1|O4?>x1 zHf4JE7@t7vK8=nZ0M#ugC#M$A2M9Z@@RG7JC2H&M0(lXnBeiwAbZBI{5AgA4FlGy= zn!;%df?G~8aW_3dOn+x!+lOTOvr#bJX!GMgLx{?u@4}&F=owr34I6Orr=SQG3_${+ z_Q!rLx`p;oggpkqCwN7O_Wu9hix@aiL?;gAJ1%F}QWKx|CK?b;ShxF2e;lC>qCN*d zVE)$?B=`9?riIe#-oUMa!8^N6X=R&5Xb^_)A~^O@n?Muf|Mimf0yJ6uW-`P>(9CDZ z(r5+tj~Irkhyna(ZiBz}<+88O?;?d(cl`Z=C<4vM9nz}TK?+h#3Vhdkm)Fmvfi+wE zrtZjqCWb#2>wiJ*00zw=qAZW`fsJN82lt;1&A0_I{UlhYJ6X}|1ZZ6KGQ0GUre*)@ z%jg`a2RbhGtuQD9NrT(Z>{7BR6)e*~*X3Y(5rZfR{r98hufaXMpOyrGL?R^so@N=_ z9U))!1GL>lIS)8kxlS*L4dkYlT6mWCa{&V?IbHj1a$%{F!C<0|v zXJrBMhO00?kAuJUUaY*kHUoU}KZr_%hSJ&2+#3LSHDfS8PfSYn3b37+O|YhYjT)EZ zNi^(F!4}x-|AZGQ4#0I_)GmMkj)Gv=E8PG&8fT$#wUeLU2hgKo@8?d@5~KgQPq2(N z^w^x$!wUeXw$Ruz}gyX`y#ZZYwH;tzod(!X#4RtR_#u;OLM_MW^Ez*KE7 z|Ng(s1PUmSgwR0>uM?U+T)7oF04*{O7)wrY8+twZe_;hQfEYMs)K3QDr2ph9ABd|) z;G9O_uV9{b7WaSQmWJy8;@T&Ge~f3Y3;>6vRbrH#=SRt))7-a*93Ih5YA@fp05a|G zq)>N=D)Xov@?2ZhdeXW6k+rGQ$AgKksG`=9p`u2=9cN|{Yi1F}+_6I6k8J4I3N79{ zl^;XOM2s}itm>5{-Xbiuw|q^zyn@F_9G!9^fpbCdM0I+f)9l`}{s(BZWwE|t*-kK9 zUv{;)`iuIo$l+pD;)PnWPsQOKmxSU(T<~_#>3csPOSO|-xe2B*inG&?vk^^6+tj?1 zWVs!)^gGJoPas~{3kV&t+vS6}z_tN{2QMY}zv3_9F>`jja60RLCJhYt==RNQ4Atf8hEX6NC9)BUf?fE*W z@%@E3{)ELB;)AXqzsI{sOO`yJjd%0tGD{ZfKDj1W6ipHOAag+%)iEEV8`U=>(`ejy zQr#s>S^ogUbm%Yqw>Bqi)~Cg}lzAx~$1(?c`lKDVuNqunbP#h}vTHBpPov(b`Bfhi zm_HnZ9}uj5UwDYJW*SJaTNqlA{g5M<`XRfP!J?}1rn;MjhH&auLvLm8SMsvKWNWEv zfVT>6E$Zytd>Ao5@U&+AnI|-)T!)xtznA=VO*-^bu={5Pd#+=Yzzo_D0rLmc6LoR6 zJjiZJCcjJz?&I^FH==pZa#eu5+(h+epRI2X8u$`iMidQKd1peVdS#YWTj^mx7_6ED z{(2{s#RB*X^xweD42FXY)ieOt?fd*Ts}L4-P;zl4Q@+2)==NAQ)!9|0NyN|Th5ITU z1JpElIUcax%K}D*Yh0M9Snr2i@4l(_upe@0Imp!~)F_g7O^v?qP z-U)g?0%`#cFFUSyhxGS}OW@{IOyPR)Sm0nlXXtj2!Oh!tX6@8BqZJA+<+wMNkzXyU zhAY$h6HYi5+x?AoZ&Z-lbKi}9fSS0%4!h6OviN0z>!u+gZF|_bcfIj;N0cnpcgCM^ zAgOS93l?(~*SDoJ9WameHuDRp`|+x&v_TjF5ovr!8shtj5$BxJv-dFwk;bmj7O!rIOe-v(^9)E>)4 z;+L(u{YlQtu_KAyug=#4gTZ7RHs;dZTNMSp9kyJH8%D(oN$5#A!+0zlSrue6?xXQe2;&b_mwhg z*XiEP&U|gA)7{EQF?$UouiV2R#XQTVyeKOc9>q@$4fsKES0kM$pEIVyJYtvqU9m0I zDg#<0%XaH&KEXw!dQo%g6*+R{n)H%Yc~AMBUeEJm%goJ@>#4Y!maQ12V^?o{8`*E@ zjUU=vpUh-qIR43;$L6QG&8low%Yi{c2JuIm5FtkR*?zuQ%rX#}JC{b}x zri9ym8G*JqXXibsIUc#7-6mY59b}y5SZ99y)rS?kJf{M)fG66o3h@c8fqff9sjgar zLSz`LzeG|HIkjNa63xWP!}9IWq6L1_CF$G|pVCjM!&M6w7Qx4!^Mg~Qdo9QG_($r`}T)^W!1)SqUBq@3IOH2>+JP?OvVBK)_v3+ztXAdPl$;73#?mtvCZ9b&z&O& zn72?{20Do^!130;touXpHvlaP2a3Ly&6Sf{TloDqo)b%|h^9hwN5K4}kIUb_&41St z@f=uwZJ<+qnPIo1cKO{*O19?ca;Nz_eT3a(;#!4KlPk# zCQ4CJC^TB4{Ab>n-Q+aGam+-#d2li}h7roaye8cI`L^qhWKTVE%4hW@LuP=P#ItI}(R;FJ@G`=6CS4y|= z4o`F};uT;XZD$vn2e-C8Or5?yj9h_RA46gcUmk$LkBgo-2i4NM zFMyl;vsR!OSBw=%-q>(!#> znd~Zd)*ch$;I|TGG{L{apGGz&XUC)oCpzHCG0$DJ@u>O7rSqi$e{V1~$kvnNN0V^H zozbc;F7u~WLr@C6^>YdXXl9-4ENPLemjDpf2ajbGK%RIh>L^I7If}4+~&|k<}yy(CtvGpUj1X9eUa{Kl=dWC(2->yKpt%<|ms5=K1Cq1O=e;0e* ze_9HDLply!i;q^+V0G{II`#2SF963ji3uKY(X5(uVklhLj61gHF9!>9*uGV>5x;Sx zW?S`yrc%ur2AEm8q`U5EAiN#dux5UyMS_w@f}-aC3wP+FTs#|J>Seq^Mx*J>K~>Da zaqZgW{3O-Dd}CMH!Qkl$bz2QC$^CUHeH$Xqqo8dm+G4w3QzxYMt7xj|fGvrMs$Mfq zntMK3JY*3nu3(FgQb#$wFbirp(v}}<{PTIEe1l9Vc|SBR=MM6T<7G&_<(ZCDmM?Cv zT|LdlXalB~^$=WNp?(Jcl5s7fR-caN^FEZ1c{pQY&BLO66j8VL1uj^trotU$lGukb z^uGtlgbfIK6)3x;F)oD0?Bf`7HR^^F)Hkbic^Ne>9?;bg$}R99HPo|H z{0DuwOMUaLj=jsNt{J_d$8ccG_Ev}XI&K~IA|s%u^E@#k;P3@Y?zZ=7h1iw)n9vW_ zOEsOC2ajD$D+(@S9FO1)Q@>UMFpY^ntp}#43Q;)+RY!%XJL5dy)UfS5k`3rPIX&R( zU5hV!)ar9p7i`HPu=UYFsnk_N4P?V>JwZEgGl6{42G1J!( zFdqV92KR60Y(|_0i|oo)ekk1$Yv4-6C)NCwi=9IXQAnMt;T2N}hPDh_vl@f8jp9WYfqtpAuX>^or^dkCks=s>H&7FrUkdxh|zV9~m9&-YgQakNzFsz{NTDn=`E}^Qr8vrE*U94Vy zgR59g+2LRV`g6VI(Eu}B_2STph0Uq5gw6@SeDRn>8!Zu~Z_yw;KAdP&$U(J0c#U7Q z<6$9dyB(x+9jK$MCpoDP8BUuR_I3!_s-RSN1`W7Q&;7tHmcCYSnkV5)o-3H28C<(d zC>p8H^}>bat}WQ{$Ntgo#o-wz*uTp>n`$WwpZyrMB)W!OMz_gPK6u6oG-bXbi7xkjb! z2kM*qQsYh=f6F89sJ_3V!^aLpI3;MS7(^>k?-m^%{dGM3u zPQ>ho<7Ff_M^vNn(o<58=dV7LV>%T-JibOy9a7#I`&PFQF%X0D1&eH0-w}4)p1PvA zy&-%BXgH9f@v^IO7{hC*Dc@5kQ;CDznFns*p$V}&DRTr^IgX-tnl1MD3ARfmL3|U! zBLUm!!zR^uzj})1`;{<0!uY$_T+PMES%IexVf)6E%Sp^O*I7)eb?^Q(C$8FgN%AHj zoss7|NX9s$Z>8`xjUj{)MfekT#Cuf3dlcREeAGf%DVg=Q%I2v*fyxlcVRhlTS zB~#$rB++xW$|5afQmj=B6(j61&wL@RkBY}_55}`S37o1Laek4xdh-k;cHstn#=vd8 zpS`*}T`a@(y9BKN+M#Nq8db)c(o@pvWy`45?YNbc)APT$ z(h}{(zz^wBegu;T7vPk0e)sc*H&LR_S_j=d-w^ea&dR6mZ0gQepa$;ZsFdh)Vb`LS z)<1`mjug<%*VG0PUCjqvPT;C#sOGOkGD=1KJm;7CrvdhecRGCJnlvfYJazTrEcH)c zoGRMaL@0i53)YGGM9JxZGOlPPLKrXhlPGez%}3aQnuIMwmwouKLdEeDG&NEM7f#qe zIihwJXK1f4#OrA&=zg^NxWjdrLu>3&OOwYBpS-EN=WA9pXb1d^a*XOooZlTKrXi3m`pat6T4^D zvqJBXva^x3-ni)^ucvB41_Q1_#p+i-)roVxct~PGSsmiJTw<8TNY8@6fL}vl>*i1u zzCZ`#Zn`XnMk0jqWg|UDxuo~^53JPG`eR*hY1C}qcpek2<2sm~AK8-Hck~-4^?9zQ z7eSX?oYY0B*1ni())nwZ?eW9Bd#tftYh_Q)E!qdFH%1+AYPq-hMC#3#)mJAXb9_Ja z$@X+QZZV`p0^{> zOZ%ul5Ey-4HKS^7!tan7w(T<_O!&fEVqMqU_B*G)k@e8E`L2<T8bQ5URXz2Y-YF^FuAOzegz|$qxzFTh z-^yF^Ln-aG!AF$bVo@@3TX|tXdbm;qrK(=(^knV-qw6ihqU^%1VJRs=fdNEHx5!qj8G2ymy||zEd4GN1|LZt7hJD6b>s;sFHX`en zod-TQ{AU7Q3{4er$4KPRjSoZBuUI0ulQd+0C%oA`#_RPtRb%CxP=a#KfGoLVYja~Q z2)1KiAk!ljgsPWyIODFrRAWIYL$S|{S#g^Ngh6Io%Rg9gR546wB#lZ?AY~kma7t+3 z88R*XYET%!Gw{G9$+k3rp{3PwVJ%;#arS510ZRUYag*kqV=Y6)ce2}Dc=tCs3;_h& z(IPk8HGJQch=Cq>%)5(Ilk)skhVX2^v@QV|?CePjI{weWXo|j86&2{v8S!a>leEqHK5IZ%#tJ01R1}|SvM0~83UXLc!=BDgN zBn*76KGzNXG+*@Tbhy~a(yg}~YpuykFq|d&w7KZgp?$Au+K1b1k+TzNtao4F^Db__ z*Pc?qUNY?sOrFU}1|)st(v!84jCtI}6yFei)Emjgag`FP(cQSh2S4^E#Yy$?_1~1* z;sNfI-j~~`S5=w4or9Y$&^=dxJ~!^F@?FTht7>-m*Z>jA3yAP*P??64HzGr-SCS)J ztbV6RaE*&yp8PUfpw)Q|a?ElXdJ7~^>2u(5^nG)x8dTI50aDJW(bAA#!?IQ@izF;B zxJ5nODl#s=bd)Q^#%@l&xZ^D$wOGH|)lVY9DO@WgF>$RL_NTvyIy%SaMU={11+&R!?SxlF(@Z+u4{0-{ zMEXj2sPQP2XMd5nLfO-IQtTd)m4@&=M1X#wavC1!Fi$+(%f0t(6#Vx?zHV3OmQ-!Z z$bHCA{TtJZSFVyi$smD;mG>r`?2&5QJ6!mrv-9k8Z?bA8mD=Owm^EA{tJ|RAM)rAy z(vYC>Z|h$<}Y+w65v#IFX zu}63DT1sd~F|W8%YZzbbs8jkF{M;4mkMtoN%qr4ReV=%-P}^}~t(9sTK^Px0uin>f z!ZhZUQ?g+Uk8**Q=!Kd+v$une3!c>89x1ZA&b0dF1+1L$`yZ>;Mt`h!5O)(rMz<-C zS&cjGwyJB^_RR1)JTv-$Sh>+xgc~@rz%=sU3Ej#AQOl_EB(YYHCu~|1v~-9sS5FwFyf4A8#=k+R*?oF*3%w`Hah?+S{cjnWNP9G2E(vLB@&}tWHXCCVkk)jS8^nH+uzDd?K(?jnqdb`MBTebwh ze%J)EyW7^)#o3Ue3nIel8&Vk%O=BsR6^yExAJx;TO|T^Rao~Vw45->5$`g6*fao)S zN0%R5)Ou82c}T3im)G%AzKf0aIo+9VnN1STz9hza^vmUvUs$F7GdDQfkf#g%BWKYo7|pHOEvPvJyGfDv7D-o58uHk z@GUg9U|~zg|3o>LzLS$6Y{IreE$^1c6nkP)SND;A^u7#+_+5<|ys!E2OL)vp{f8|f z)O(nU@lBKvP1QKF&&=Mw%B@X9{@bNMuZ&2&c3~!AoAQEi64o2(wey zW4|bYj_#9syF$*pZp7ewUmIb~oy<4S#U@E!P}wje zHl*J(?ymGrvdm|mem+_#A>GV}1(sp(RraFAT+DYJRjzvKushZ>wE8qDk!Tv{+y^ex<&ozZsi~$)}_%+BA_KWsKu3|Niw8qGAn+$ z8DrUVeH^=jMDQ=zRYQ1?OOfwI|AI3|QZyF_0)0|sL(cfJ&JgZK6tH4S&J5Ez;VT6N zAw+*yyzokkc+(FS&_wKI@F%JZ4=ucg=gAW3`njoEzV>Gz|Fz4a$x5aKywFc?{yiXn z3NsV#`sX%K`$aY23IEtZM_%F89VGc!G#B&kUTx{?4y>Y(D#CX$Y6a{d$JS~bk~1q3 z9~Wa<b>TE%R#2rjF~8z9{vf}NnO`lE1OMqB`)YTz*oz9+yR7RM1B&t zT+PW!Ze4bQZFFXiN5BMvPshUiFpHV)C#KQ;tc$;46&MGj%Do+|yM8JpT=ZPSDVU%q zuV4K4TJ_clTsrF)<%ThSR(0LE{-gois_f^@IPb6Mfv9nn)*7!^%r1twXi zlrrDUS>^X`q`4zxkOY&6wKmkP#l_H%CmD6qXy0ORBE0U$i5@DG*B!9+{7(S&*XHRsk&9+Sn+vZ$7^LW&z;et{i;3E{5!3k>sUJpHs6(-ocC|k`#(hxPKM0K zkURX_*>9)jfz|g6V&>de9E$m6K6_Zln6!o)6fq`FRr!Ye;wQWP$+CZzFLGFR;JSA` zEM&PiBGQDN6Q8v)_G555KGUV<5wzxbZ~c%l$1K5h zS1#}Kszh2kv@u$lgv<_Eco5&m)~V#K8On7fb^oQLMvJ>`avGf3Ghc;UXY2nP_M8(Y z;)*RZsoBAA=Fj(yq4qFbE#Uj~js2ka!<_4ja_ort(#Bawvk`Pye9C zR2%R)`T=vhgh@W1)@3T1z1~(oSn*#GKcBBVe<&%R5INNDnoE&%&T1yjlJrnZvw5v# z5mqBHbL;ijnlp)LAd@i6ChtIHu58q+y4^4swdum!ILGP^m-3+CBqZq?m)=nJWfahj z=jwQbuZa>iFV1?j16k`PylRBcjb@kd>aEdde_X5lwT>sF(VQC$6g^?~5s(vJa`V$g z4!9JpiQNtML3X}JA6)Q`H9)WWu9sHQQGbD`ZIG-#PJ>txJL|whw8nTWF1bKrFnvar5!DWnirtV3SqMX9@vf{*NLTGCJ+xA$Ohep zmItc2L|NrtBJL+nTP1ysH&40@j~zXTnNkU!8veG?H+ShRKSEwyI+{V6@&ZvBm%sDG z^5MvU7Aluy`6hb;;eoU$QC}QaC~5xUewBvk)$a`U%5}!Bp12Df)hTbD+vae&9Rc|F zFS;vQ+bJ}ZmJS`{0yre_g*R*Zw6-o9-e3(I>^n<$GrFJNDbj%)7tq_Fo205D}&7p3vGr z9Fun?Q)lD&lyQVOHibKF+e1s}5M?-mpmam0ZwD?t??c@t3u49%uKE<#BB7&b7IYXx zWHw!?A(fI|RbSar;%zjEiN390zne-5sY4;X)s&wPBN!HePdArGcidU&(KL(zrr z#&!PMeg_-ZKnIzy)ROBJuXU9ytl`8`_D*E=M?&-msqku)QNZ5mUxbqA7Eay3aX|l3=om{gvd!{m@ zeo_&SR$U%g`G7EQBlJ6$f9HtP$z;jm$x4pbh9<#q!yYf~b$6>t$Jt#u+L%e@2`#8b zZqy}wq(p6{EJXz$i!NWSacLI9kgHRof>9fVoN_ZRQf9k8KBM{R_aNeRVUqF@ zrr9bo>()(bJfU|LIKf~EIV!#^dE@9yU}`r2;$}BM@Roq^%%mYg19gk0xVR?^$-l|U z-8LwSUd81e7PdrcT^aS+<0C&Zm5vhplW0*$&DA8NsN#$ET>2-JRjlhXy}gBr8T3+< z20k^v!+mHNk+`rOE78rBIB~mNNprapRW~7U=uq@9zl&M6=NY~JC4z_kk2l{WasRSM zcra=*oZ*V9zLHwut!i2yVwrgLT#VVh+-{b-tpD#?jt+BdsvRerr~#c97RHFcq>$H? z&~SZ}*vB?PHCRg`S~iC5ZwVV{W8J`hPV&<@Mje^Qm}YA7d$^n|tP0zo`|fF$Jo@U> zisgu=rTv(Xzn9xTht69#gc+oF$2XU;@FBesWEI2lLxy1>83@{|`5;uzF^zGEH7>`= zxet`Yh