Merge branch 'v4' of github.com:jumpserver/jumpserver into v4

pull/13155/head
ibuler 2024-04-29 15:07:25 +08:00
commit c05a3c315a
298 changed files with 15234 additions and 2908 deletions

View File

@ -1,11 +1,35 @@
---
name: 需求建议
about: 提出针对本项目的想法和建议
title: "[Feature] "
title: "[Feature] 需求标题"
labels: 类型:需求
assignees:
- ibuler
- baijiangjie
---
**请描述您的需求或者改进建议.**
## 注意
_针对过于简单的需求描述不予考虑。请确保提供足够的细节和信息以支持功能的开发和实现。_
## 功能名称
[在这里输入功能的名称或标题]
## 功能描述
[在这里描述该功能的详细内容,包括其作用、目的和所需的功能]
## 用户故事(可选)
[如果适用,可以提供用户故事来更好地理解该功能的使用场景和用户期望]
## 功能要求
- [要求1描述该功能的具体要求如界面设计、交互逻辑等]
- [要求2描述该功能的另一个具体要求]
- [以此类推,列出所有相关的功能要求]
## 示例或原型(可选)
[如果有的话,提供该功能的示例或原型图以更好地说明功能的实现方式]
## 优先级
[描述该功能的优先级,如高、中、低,或使用数字等其他标识]
## 备注(可选)
[在这里添加任何其他相关信息或备注]

View File

@ -1,22 +1,51 @@
---
name: Bug 提交
about: 提交产品缺陷帮助我们更好的改进
title: "[Bug] "
title: "[Bug] Bug 标题"
labels: 类型:Bug
assignees:
- baijiangjie
---
**JumpServer 版本( v2.28 之前的版本不再支持 )**
## 注意
**JumpServer 版本( v2.28 之前的版本不再支持 )** <br>
_针对过于简单的 Bug 描述不予考虑。请确保提供足够的细节和信息以支持 Bug 的复现和修复。_
## 当前使用的 JumpServer 版本 (必填)
[在这里输入当前使用的 JumpServer 的版本号]
## 使用的版本类型 (必填)
- [ ] 社区版
- [ ] 企业版
- [ ] 企业试用版
**浏览器版本**
## 版本安装方式 (必填)
- [ ] 在线安装 (一键命令)
- [ ] 离线安装 (下载离线包)
- [ ] All-in-One
- [ ] 1Panel 安装
- [ ] Kubernetes 安装
- [ ] 源码安装
## Bug 描述 (详细)
[在这里描述 Bug 的详细情况,包括其影响和出现的具体情况]
**Bug 描述**
## 复现步骤
1. [描述如何复现 Bug 的第一步]
2. [描述如何复现 Bug 的第二步]
3. [以此类推,列出所有复现 Bug 所需的步骤]
## 期望行为
[描述 Bug 出现时期望的系统行为或结果]
**Bug 重现步骤(有截图更好)**
1.
2.
3.
## 实际行为
[描述实际上发生了什么,以及 Bug 出现的具体情况]
## 系统环境
- 操作系统:[例如Windows 10, macOS Big Sur]
- 浏览器/应用版本:[如果适用,请提供相关版本信息]
- 其他相关环境信息:[如果有其他相关环境信息,请在此处提供]
## 附加信息(可选)
[在这里添加任何其他相关信息,如截图、错误信息等]

View File

@ -1,10 +1,50 @@
---
name: 问题咨询
about: 提出针对本项目安装部署、使用及其他方面的相关问题
title: "[Question] "
title: "[Question] 问题标题"
labels: 类型:提问
assignees:
- baijiangjie
---
## 注意
**请描述您的问题.** <br>
**JumpServer 版本( v2.28 之前的版本不再支持 )** <br>
_针对过于简单的 Bug 描述不予考虑。请确保提供足够的细节和信息以支持 Bug 的复现和修复。_
## 当前使用的 JumpServer 版本 (必填)
[在这里输入当前使用的 JumpServer 的版本号]
## 使用的版本类型 (必填)
- [ ] 社区版
- [ ] 企业版
- [ ] 企业试用版
## 版本安装方式 (必填)
- [ ] 在线安装 (一键命令)
- [ ] 离线安装 (下载离线包)
- [ ] All-in-One
- [ ] 1Panel 安装
- [ ] Kubernetes 安装
- [ ] 源码安装
## 问题描述 (详细)
[在这里描述你遇到的问题]
## 背景信息
- 操作系统:[例如Windows 10, macOS Big Sur]
- 浏览器/应用版本:[如果适用,请提供相关版本信息]
- 其他相关环境信息:[如果有其他相关环境信息,请在此处提供]
## 具体问题
[在这里详细描述你的问题,包括任何相关细节或错误信息]
## 尝试过的解决方法
[如果你已经尝试过解决问题,请在这里列出你已经尝试过的解决方法]
## 预期结果
[描述你期望的解决方案或结果]
## 我们的期望
[描述你希望我们提供的帮助或支持]
**请描述您的问题.**

1
.gitignore vendored
View File

@ -43,3 +43,4 @@ releashe
data/*
test.py
.history/
.test/

View File

@ -39,6 +39,23 @@ ARG BUILD_DEPENDENCIES=" \
pkg-config"
ARG DEPENDENCIES=" \
freetds-dev \
libffi-dev \
libjpeg-dev \
libkrb5-dev \
libldap2-dev \
libpq-dev \
libsasl2-dev \
libssl-dev \
libxml2-dev \
libxmlsec1-dev \
libxmlsec1-openssl \
freerdp2-dev \
libaio-dev"
ARG TOOLS=" \
ca-certificates \
curl \
default-libmysqlclient-dev \
default-mysql-client \
libldap2-dev \
@ -77,6 +94,7 @@ ENV LANG=en_US.UTF-8 \
PATH=/opt/py3/bin:$PATH
ARG DEPENDENCIES=" \
libjpeg-dev \
libldap2-dev \
libpq-dev \
libx11-dev \
@ -85,6 +103,11 @@ ARG DEPENDENCIES=" \
ARG TOOLS=" \
ca-certificates \
default-libmysqlclient-dev \
default-mysql-client \
iputils-ping \
locales \
netcat-openbsd \
nmap \
openssh-client \
sshpass"
@ -103,9 +126,18 @@ RUN --mount=type=cache,target=/var/cache/apt,sharing=locked,id=core-apt \
&& sed -i "s@# export @export @g" ~/.bashrc \
&& sed -i "s@# alias @alias @g" ~/.bashrc
ARG RECEPTOR_VERSION=v1.4.5
RUN set -ex \
&& wget -O /opt/receptor.tar.gz https://github.com/ansible/receptor/releases/download/${RECEPTOR_VERSION}/receptor_${RECEPTOR_VERSION/v/}_linux_${TARGETARCH}.tar.gz \
&& tar -xf /opt/receptor.tar.gz -C /usr/local/bin/ \
&& chown root:root /usr/local/bin/receptor \
&& chmod 755 /usr/local/bin/receptor \
&& rm -f /opt/receptor.tar.gz
COPY --from=stage-2 /opt/py3 /opt/py3
COPY --from=stage-1 /usr/local/bin /usr/local/bin
COPY --from=stage-1 /opt/jumpserver/release/jumpserver /opt/jumpserver
COPY --from=stage-1 /opt/jumpserver/release/jumpserver/apps/libs/ansible/ansible.cfg /etc/ansible/
WORKDIR /opt/jumpserver

View File

@ -85,7 +85,7 @@ If you find a security problem, please contact us directly
- 400-052-0755
### License & Copyright
Copyright (c) 2014-2022 FIT2CLOUD Tech, Inc., All rights reserved.
Copyright (c) 2014-2024 FIT2CLOUD Tech, Inc., All rights reserved.
Licensed under The GNU General Public License version 3 (GPLv3) (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

View File

@ -18,9 +18,8 @@ __all__ = [
class AccountBackupPlanViewSet(OrgBulkModelViewSet):
model = AccountBackupAutomation
filter_fields = ('name',)
search_fields = filter_fields
ordering = ('name',)
filterset_fields = ('name',)
search_fields = filterset_fields
serializer_class = serializers.AccountBackupSerializer

View File

@ -20,8 +20,8 @@ __all__ = [
class AutomationAssetsListApi(generics.ListAPIView):
model = BaseAutomation
serializer_class = serializers.AutomationAssetsSerializer
filter_fields = ("name", "address")
search_fields = filter_fields
filterset_fields = ("name", "address")
search_fields = filterset_fields
def get_object(self):
pk = self.kwargs.get('pk')

View File

@ -6,9 +6,12 @@ from rest_framework.response import Response
from accounts import serializers
from accounts.const import AutomationTypes
from accounts.filters import ChangeSecretRecordFilterSet
from accounts.models import ChangeSecretAutomation, ChangeSecretRecord
from accounts.tasks import execute_automation_record_task
from authentication.permissions import UserConfirmation, ConfirmType
from orgs.mixins.api import OrgBulkModelViewSet, OrgGenericViewSet
from rbac.permissions import RBACPermission
from .base import (
AutomationAssetsListApi, AutomationRemoveAssetApi, AutomationAddAssetApi,
AutomationNodeAddRemoveApi, AutomationExecutionViewSet
@ -24,35 +27,54 @@ __all__ = [
class ChangeSecretAutomationViewSet(OrgBulkModelViewSet):
model = ChangeSecretAutomation
filter_fields = ('name', 'secret_type', 'secret_strategy')
search_fields = filter_fields
filterset_fields = ('name', 'secret_type', 'secret_strategy')
search_fields = filterset_fields
serializer_class = serializers.ChangeSecretAutomationSerializer
class ChangeSecretRecordViewSet(mixins.ListModelMixin, OrgGenericViewSet):
serializer_class = serializers.ChangeSecretRecordSerializer
filterset_fields = ('asset_id', 'execution_id')
filterset_class = ChangeSecretRecordFilterSet
search_fields = ('asset__address',)
tp = AutomationTypes.change_secret
serializer_classes = {
'default': serializers.ChangeSecretRecordSerializer,
'secret': serializers.ChangeSecretRecordViewSecretSerializer,
}
rbac_perms = {
'execute': 'accounts.add_changesecretexecution',
'secret': 'accounts.view_changesecretrecord',
}
def get_permissions(self):
if self.action == 'secret':
self.permission_classes = [
RBACPermission,
UserConfirmation.require(ConfirmType.MFA)
]
return super().get_permissions()
def get_queryset(self):
return ChangeSecretRecord.objects.all()
@action(methods=['post'], detail=False, url_path='execute')
def execute(self, request, *args, **kwargs):
record_id = request.data.get('record_id')
record = self.get_queryset().filter(pk=record_id)
if not record:
record_ids = request.data.get('record_ids')
records = self.get_queryset().filter(id__in=record_ids)
execution_count = records.values_list('execution_id', flat=True).distinct().count()
if execution_count != 1:
return Response(
{'detail': 'record not found'},
status=status.HTTP_404_NOT_FOUND
{'detail': 'Only one execution is allowed to execute'},
status=status.HTTP_400_BAD_REQUEST
)
task = execute_automation_record_task.delay(record_id, self.tp)
task = execute_automation_record_task.delay(record_ids, self.tp)
return Response({'task': task.id}, status=status.HTTP_200_OK)
@action(methods=['get'], detail=True, url_path='secret')
def secret(self, request, *args, **kwargs):
instance = self.get_object()
serializer = self.get_serializer(instance)
return Response(serializer.data)
class ChangSecretExecutionViewSet(AutomationExecutionViewSet):
rbac_perms = (

View File

@ -20,8 +20,8 @@ __all__ = [
class GatherAccountsAutomationViewSet(OrgBulkModelViewSet):
model = GatherAccountsAutomation
filter_fields = ('name',)
search_fields = filter_fields
filterset_fields = ('name',)
search_fields = filterset_fields
serializer_class = serializers.GatherAccountAutomationSerializer

View File

@ -20,8 +20,8 @@ __all__ = [
class PushAccountAutomationViewSet(OrgBulkModelViewSet):
model = PushAccountAutomation
filter_fields = ('name', 'secret_type', 'secret_strategy')
search_fields = filter_fields
filterset_fields = ('name', 'secret_type', 'secret_strategy')
search_fields = filterset_fields
serializer_class = serializers.PushAccountAutomationSerializer

View File

@ -6,7 +6,7 @@ from django.conf import settings
from rest_framework import serializers
from xlsxwriter import Workbook
from accounts.const.automation import AccountBackupType
from accounts.const import AccountBackupType
from accounts.models.automations.backup_account import AccountBackupAutomation
from accounts.notifications import AccountBackupExecutionTaskMsg, AccountBackupByObjStorageExecutionTaskMsg
from accounts.serializers import AccountSecretSerializer
@ -168,9 +168,8 @@ class AccountBackupHandler:
if not user.secret_key:
attachment_list = []
else:
password = user.secret_key.encode('utf8')
attachment = os.path.join(PATH, f'{plan_name}-{local_now_filename()}-{time.time()}.zip')
encrypt_and_compress_zip_file(attachment, password, files)
encrypt_and_compress_zip_file(attachment, user.secret_key, files)
attachment_list = [attachment, ]
AccountBackupExecutionTaskMsg(plan_name, user).publish(attachment_list)
print('邮件已发送至{}({})'.format(user, user.email))
@ -191,7 +190,6 @@ class AccountBackupHandler:
attachment = os.path.join(PATH, f'{plan_name}-{local_now_filename()}-{time.time()}.zip')
if password:
print('\033[32m>>> 使用加密密码对文件进行加密中\033[0m')
password = password.encode('utf8')
encrypt_and_compress_zip_file(attachment, password, files)
else:
zip_files(attachment, files)

View File

@ -18,6 +18,8 @@
become_user: "{{ custom_become_user | default('') }}"
become_password: "{{ custom_become_password | default('') }}"
become_private_key_path: "{{ custom_become_private_key_path | default(None) }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"
register: ping_info
delegate_to: localhost
@ -54,4 +56,6 @@
become_user: "{{ account.become.ansible_user | default('') }}"
become_password: "{{ account.become.ansible_password | default('') }}"
become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"
delegate_to: localhost

View File

@ -7,6 +7,7 @@ type:
- all
method: change_secret
protocol: ssh
priority: 50
params:
- name: commands
type: list

View File

@ -85,6 +85,7 @@
become_user: "{{ account.become.ansible_user | default('') }}"
become_password: "{{ account.become.ansible_password | default('') }}"
become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
when: account.secret_type == "password"
delegate_to: localhost
@ -95,5 +96,6 @@
login_user: "{{ account.username }}"
login_private_key_path: "{{ account.private_key_path }}"
gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
when: account.secret_type == "ssh_key"
delegate_to: localhost

View File

@ -85,6 +85,7 @@
become_user: "{{ account.become.ansible_user | default('') }}"
become_password: "{{ account.become.ansible_password | default('') }}"
become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
when: account.secret_type == "password"
delegate_to: localhost
@ -95,5 +96,6 @@
login_user: "{{ account.username }}"
login_private_key_path: "{{ account.private_key_path }}"
gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
when: account.secret_type == "ssh_key"
delegate_to: localhost

View File

@ -5,6 +5,7 @@ method: change_secret
category: host
type:
- windows
priority: 49
params:
- name: groups
type: str

View File

@ -7,9 +7,9 @@ from django.utils import timezone
from django.utils.translation import gettext_lazy as _
from xlsxwriter import Workbook
from accounts.const import AutomationTypes, SecretType, SSHKeyStrategy, SecretStrategy
from accounts.const import AutomationTypes, SecretType, SSHKeyStrategy, SecretStrategy, ChangeSecretRecordStatusChoice
from accounts.models import ChangeSecretRecord
from accounts.notifications import ChangeSecretExecutionTaskMsg
from accounts.notifications import ChangeSecretExecutionTaskMsg, ChangeSecretFailedMsg
from accounts.serializers import ChangeSecretRecordBackUpSerializer
from assets.const import HostTypes
from common.utils import get_logger
@ -27,7 +27,7 @@ class ChangeSecretManager(AccountBasePlaybookManager):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.record_id = self.execution.snapshot.get('record_id')
self.record_map = self.execution.snapshot.get('record_map', {})
self.secret_type = self.execution.snapshot.get('secret_type')
self.secret_strategy = self.execution.snapshot.get(
'secret_strategy', SecretStrategy.custom
@ -119,14 +119,24 @@ class ChangeSecretManager(AccountBasePlaybookManager):
else:
new_secret = self.get_secret(secret_type)
if self.record_id is None:
if new_secret is None:
print(f'new_secret is None, account: {account}')
continue
asset_account_id = f'{asset.id}-{account.id}'
if asset_account_id not in self.record_map:
recorder = ChangeSecretRecord(
asset=asset, account=account, execution=self.execution,
old_secret=account.secret, new_secret=new_secret,
)
records.append(recorder)
else:
recorder = ChangeSecretRecord.objects.get(id=self.record_id)
record_id = self.record_map[asset_account_id]
try:
recorder = ChangeSecretRecord.objects.get(id=record_id)
except ChangeSecretRecord.DoesNotExist:
print(f"Record {record_id} not found")
continue
self.name_recorder_mapper[h['name']] = recorder
@ -154,25 +164,43 @@ class ChangeSecretManager(AccountBasePlaybookManager):
recorder = self.name_recorder_mapper.get(host)
if not recorder:
return
recorder.status = 'success'
recorder.status = ChangeSecretRecordStatusChoice.success.value
recorder.date_finished = timezone.now()
recorder.save()
account = recorder.account
if not account:
print("Account not found, deleted ?")
return
account.secret = recorder.new_secret
account.date_updated = timezone.now()
account.save(update_fields=['secret', 'date_updated'])
max_retries = 3
retry_count = 0
while retry_count < max_retries:
try:
recorder.save()
account.save(update_fields=['secret', 'version', 'date_updated'])
break
except Exception as e:
retry_count += 1
if retry_count == max_retries:
self.on_host_error(host, str(e), result)
else:
print(f'retry {retry_count} times for {host} recorder save error: {e}')
time.sleep(1)
def on_host_error(self, host, error, result):
recorder = self.name_recorder_mapper.get(host)
if not recorder:
return
recorder.status = 'failed'
recorder.status = ChangeSecretRecordStatusChoice.failed.value
recorder.date_finished = timezone.now()
recorder.error = error
recorder.save()
try:
recorder.save()
except Exception as e:
print(f"\033[31m Save {host} recorder error: {e} \033[0m\n")
def on_runner_failed(self, runner, e):
logger.error("Account error: ", e)
@ -188,7 +216,7 @@ class ChangeSecretManager(AccountBasePlaybookManager):
def get_summary(recorders):
total, succeed, failed = 0, 0, 0
for recorder in recorders:
if recorder.status == 'success':
if recorder.status == ChangeSecretRecordStatusChoice.success.value:
succeed += 1
else:
failed += 1
@ -205,18 +233,35 @@ class ChangeSecretManager(AccountBasePlaybookManager):
summary = self.get_summary(recorders)
print(summary, end='')
if self.record_id:
if self.record_map:
return
self.send_recorder_mail(recorders, summary)
failed_recorders = [
r for r in recorders
if r.status == ChangeSecretRecordStatusChoice.failed.value
]
def send_recorder_mail(self, recorders, summary):
recipients = self.execution.recipients
if not recorders or not recipients:
recipients = User.objects.filter(id__in=list(recipients.keys()))
if not recipients:
return
recipients = User.objects.filter(id__in=list(recipients.keys()))
if failed_recorders:
name = self.execution.snapshot.get('name')
execution_id = str(self.execution.id)
_ids = [r.id for r in failed_recorders]
asset_account_errors = ChangeSecretRecord.objects.filter(
id__in=_ids).values_list('asset__name', 'account__username', 'error')
for user in recipients:
ChangeSecretFailedMsg(name, execution_id, user, asset_account_errors).publish()
if not recorders:
return
self.send_recorder_mail(recipients, recorders, summary)
def send_recorder_mail(self, recipients, recorders, summary):
name = self.execution.snapshot['name']
path = os.path.join(os.path.dirname(settings.BASE_DIR), 'tmp')
filename = os.path.join(path, f'{name}-{local_now_filename()}-{time.time()}.xlsx')
@ -226,9 +271,8 @@ class ChangeSecretManager(AccountBasePlaybookManager):
for user in recipients:
attachments = []
if user.secret_key:
password = user.secret_key.encode('utf8')
attachment = os.path.join(path, f'{name}-{local_now_filename()}-{time.time()}.zip')
encrypt_and_compress_zip_file(attachment, password, [filename])
encrypt_and_compress_zip_file(attachment, user.secret_key, [filename])
attachments = [attachment]
ChangeSecretExecutionTaskMsg(name, user, summary).publish(attachments)
os.remove(filename)

View File

@ -58,7 +58,7 @@ class GatherAccountsManager(AccountBasePlaybookManager):
result = self.filter_success_result(asset.type, info)
self.collect_asset_account_info(asset, result)
else:
logger.error(f'Not found {host} info')
print(f'\033[31m Not found {host} info \033[0m\n')
def update_or_create_accounts(self):
for asset, data in self.asset_account_info.items():

View File

@ -85,6 +85,7 @@
become_user: "{{ account.become.ansible_user | default('') }}"
become_password: "{{ account.become.ansible_password | default('') }}"
become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
when: account.secret_type == "password"
delegate_to: localhost
@ -95,6 +96,7 @@
login_user: "{{ account.username }}"
login_private_key_path: "{{ account.private_key_path }}"
gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
when: account.secret_type == "ssh_key"
delegate_to: localhost

View File

@ -85,6 +85,7 @@
become_user: "{{ account.become.ansible_user | default('') }}"
become_password: "{{ account.become.ansible_password | default('') }}"
become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
when: account.secret_type == "password"
delegate_to: localhost
@ -95,6 +96,7 @@
login_user: "{{ account.username }}"
login_private_key_path: "{{ account.private_key_path }}"
gateway_args: "{{ jms_asset.ansible_ssh_common_args | default('') }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
when: account.secret_type == "ssh_key"
delegate_to: localhost

View File

@ -5,6 +5,7 @@ method: push_account
category: host
type:
- windows
priority: 49
params:
- name: groups
type: str

View File

@ -60,8 +60,11 @@ class RemoveAccountManager(AccountBasePlaybookManager):
if not tuple_asset_gather_account:
return
asset, gather_account = tuple_asset_gather_account
Account.objects.filter(
asset_id=asset.id,
username=gather_account.username
).delete()
gather_account.delete()
try:
Account.objects.filter(
asset_id=asset.id,
username=gather_account.username
).delete()
gather_account.delete()
except Exception as e:
print(f'\033[31m Delete account {gather_account.username} failed: {e} \033[0m\n')

View File

@ -3,6 +3,7 @@
vars:
ansible_shell_type: sh
ansible_connection: local
ansible_python_interpreter: /opt/py3/bin/python
tasks:
- name: Verify account (pyfreerdp)

View File

@ -6,6 +6,7 @@ type:
- windows
method: verify_account
protocol: rdp
priority: 1
i18n:
Windows rdp account verify:

View File

@ -19,3 +19,5 @@
become_user: "{{ account.become.ansible_user | default('') }}"
become_password: "{{ account.become.ansible_password | default('') }}"
become_private_key_path: "{{ account.become.ansible_ssh_private_key_file | default(None) }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"

View File

@ -7,6 +7,7 @@ type:
- all
method: verify_account
protocol: ssh
priority: 50
i18n:
SSH account verify:

View File

@ -51,6 +51,9 @@ class VerifyAccountManager(AccountBasePlaybookManager):
h['name'] += '(' + account.username + ')'
self.host_account_mapper[h['name']] = account
secret = account.secret
if secret is None:
print(f'account {account.name} secret is None')
continue
private_key_path = None
if account.secret_type == SecretType.SSH_KEY:
@ -62,7 +65,7 @@ class VerifyAccountManager(AccountBasePlaybookManager):
'name': account.name,
'username': account.username,
'secret_type': account.secret_type,
'secret': account.escape_jinja2_syntax(secret),
'secret': account.escape_jinja2_syntax(secret),
'private_key_path': private_key_path,
'become': account.get_ansible_become_auth(),
}
@ -73,8 +76,14 @@ class VerifyAccountManager(AccountBasePlaybookManager):
def on_host_success(self, host, result):
account = self.host_account_mapper.get(host)
account.set_connectivity(Connectivity.OK)
try:
account.set_connectivity(Connectivity.OK)
except Exception as e:
print(f'\033[31m Update account {account.name} connectivity failed: {e} \033[0m\n')
def on_host_error(self, host, error, result):
account = self.host_account_mapper.get(host)
account.set_connectivity(Connectivity.ERR)
try:
account.set_connectivity(Connectivity.ERR)
except Exception as e:
print(f'\033[31m Update account {account.name} connectivity failed: {e} \033[0m\n')

View File

@ -15,6 +15,7 @@ class AliasAccount(TextChoices):
INPUT = '@INPUT', _('Manual input')
USER = '@USER', _('Dynamic user')
ANON = '@ANON', _('Anonymous account')
SPEC = '@SPEC', _('Specified account')
@classmethod
def virtual_choices(cls):

View File

@ -16,7 +16,7 @@ DEFAULT_PASSWORD_RULES = {
__all__ = [
'AutomationTypes', 'SecretStrategy', 'SSHKeyStrategy', 'Connectivity',
'DEFAULT_PASSWORD_LENGTH', 'DEFAULT_PASSWORD_RULES', 'TriggerChoice',
'PushAccountActionChoice', 'AccountBackupType'
'PushAccountActionChoice', 'AccountBackupType', 'ChangeSecretRecordStatusChoice',
]
@ -103,3 +103,9 @@ class AccountBackupType(models.TextChoices):
email = 'email', _('Email')
# 目前只支持sftp方式
object_storage = 'object_storage', _('SFTP')
class ChangeSecretRecordStatusChoice(models.TextChoices):
failed = 'failed', _('Failed')
success = 'success', _('Success')
pending = 'pending', _('Pending')

View File

@ -5,7 +5,7 @@ from django_filters import rest_framework as drf_filters
from assets.models import Node
from common.drf.filters import BaseFilterSet
from .models import Account, GatheredAccount
from .models import Account, GatheredAccount, ChangeSecretRecord
class AccountFilterSet(BaseFilterSet):
@ -52,6 +52,7 @@ class AccountFilterSet(BaseFilterSet):
class GatheredAccountFilterSet(BaseFilterSet):
node_id = drf_filters.CharFilter(method='filter_nodes')
asset_id = drf_filters.CharFilter(field_name='asset_id', lookup_expr='exact')
asset_name = drf_filters.CharFilter(field_name='asset__name', lookup_expr='icontains')
@staticmethod
def filter_nodes(queryset, name, value):
@ -60,3 +61,13 @@ class GatheredAccountFilterSet(BaseFilterSet):
class Meta:
model = GatheredAccount
fields = ['id', 'username']
class ChangeSecretRecordFilterSet(BaseFilterSet):
asset_name = drf_filters.CharFilter(field_name='asset__name', lookup_expr='icontains')
account_username = drf_filters.CharFilter(field_name='account__username', lookup_expr='icontains')
execution_id = drf_filters.CharFilter(field_name='execution_id', lookup_expr='exact')
class Meta:
model = ChangeSecretRecord
fields = ['id', 'status', 'asset_id', 'execution']

View File

@ -1,8 +1,9 @@
# Generated by Django 4.1.10 on 2023-08-01 09:12
from django.db import migrations, models
import uuid
from django.db import migrations, models
class Migration(migrations.Migration):
@ -20,7 +21,7 @@ class Migration(migrations.Migration):
('date_updated', models.DateTimeField(auto_now=True, verbose_name='Date updated')),
('id', models.UUIDField(default=uuid.uuid4, primary_key=True, serialize=False)),
('org_id', models.CharField(blank=True, db_index=True, default='', max_length=36, verbose_name='Organization')),
('alias', models.CharField(choices=[('@INPUT', 'Manual input'), ('@USER', 'Dynamic user'), ('@ANON', 'Anonymous account')], max_length=128, verbose_name='Alias')),
('alias', models.CharField(choices=[('@INPUT', 'Manual input'), ('@USER', 'Dynamic user'), ('@ANON', 'Anonymous account'), ('@SPEC', 'Specified account')], max_length=128, verbose_name='Alias')),
('secret_from_login', models.BooleanField(default=None, null=True, verbose_name='Secret from login')),
],
options={

View File

@ -8,7 +8,7 @@ from django.db import models
from django.db.models import F
from django.utils.translation import gettext_lazy as _
from accounts.const.automation import AccountBackupType
from accounts.const import AccountBackupType
from common.const.choices import Trigger
from common.db import fields
from common.db.encoder import ModelJSONFieldEncoder

View File

@ -2,7 +2,7 @@ from django.db import models
from django.utils.translation import gettext_lazy as _
from accounts.const import (
AutomationTypes
AutomationTypes, ChangeSecretRecordStatusChoice
)
from common.db import fields
from common.db.models import JMSBaseModel
@ -40,7 +40,10 @@ class ChangeSecretRecord(JMSBaseModel):
new_secret = fields.EncryptTextField(blank=True, null=True, verbose_name=_('New secret'))
date_started = models.DateTimeField(blank=True, null=True, verbose_name=_('Date started'))
date_finished = models.DateTimeField(blank=True, null=True, verbose_name=_('Date finished'))
status = models.CharField(max_length=16, default='pending', verbose_name=_('Status'))
status = models.CharField(
max_length=16, verbose_name=_('Status'),
default=ChangeSecretRecordStatusChoice.pending.value
)
error = models.TextField(blank=True, null=True, verbose_name=_('Error'))
class Meta:

View File

@ -137,16 +137,13 @@ class BaseAccount(VaultModelMixin, JMSOrgBaseModel):
else:
return None
@property
def private_key_path(self):
def get_private_key_path(self, path):
if self.secret_type != SecretType.SSH_KEY \
or not self.secret \
or not self.private_key:
return None
project_dir = settings.PROJECT_DIR
tmp_dir = os.path.join(project_dir, 'tmp')
key_name = '.' + md5(self.private_key.encode('utf-8')).hexdigest()
key_path = os.path.join(tmp_dir, key_name)
key_path = os.path.join(path, key_name)
if not os.path.exists(key_path):
# https://github.com/ansible/ansible-runner/issues/544
# ssh requires OpenSSH format keys to have a full ending newline.
@ -158,6 +155,12 @@ class BaseAccount(VaultModelMixin, JMSOrgBaseModel):
os.chmod(key_path, 0o400)
return key_path
@property
def private_key_path(self):
project_dir = settings.PROJECT_DIR
tmp_dir = os.path.join(project_dir, 'tmp')
return self.get_private_key_path(tmp_dir)
def get_private_key(self):
if not self.private_key:
return None

View File

@ -1,6 +1,7 @@
from django.template.loader import render_to_string
from django.utils.translation import gettext_lazy as _
from accounts.models import ChangeSecretRecord
from common.tasks import send_mail_attachment_async, upload_backup_to_obj_storage
from notifications.notifications import UserMessage
from terminal.models.component.storage import ReplayStorage
@ -98,3 +99,35 @@ class GatherAccountChangeMsg(UserMessage):
def gen_test_msg(cls):
user = User.objects.first()
return cls(user, {})
class ChangeSecretFailedMsg(UserMessage):
subject = _('Change secret or push account failed information')
def __init__(self, name, execution_id, user, asset_account_errors: list):
self.name = name
self.execution_id = execution_id
self.asset_account_errors = asset_account_errors
super().__init__(user)
def get_html_msg(self) -> dict:
context = {
'name': self.name,
'recipient': self.user,
'execution_id': self.execution_id,
'asset_account_errors': self.asset_account_errors
}
message = render_to_string('accounts/change_secret_failed_info.html', context)
return {
'subject': str(self.subject),
'message': message
}
@classmethod
def gen_test_msg(cls):
name = 'test'
user = User.objects.first()
record = ChangeSecretRecord.objects.first()
execution_id = str(record.execution_id)
return cls(name, execution_id, user, [])

View File

@ -21,6 +21,7 @@ __all__ = [
class BaseAutomationSerializer(PeriodTaskSerializerMixin, BulkOrgResourceModelSerializer):
assets = ObjectRelatedField(many=True, required=False, queryset=Asset.objects, label=_('Assets'))
nodes = ObjectRelatedField(many=True, required=False, queryset=Node.objects, label=_('Nodes'))
is_periodic = serializers.BooleanField(default=False, required=False, label=_("Periodic perform"))
class Meta:
read_only_fields = [

View File

@ -4,7 +4,8 @@ from django.utils.translation import gettext_lazy as _
from rest_framework import serializers
from accounts.const import (
AutomationTypes, SecretType, SecretStrategy, SSHKeyStrategy
AutomationTypes, SecretType, SecretStrategy,
SSHKeyStrategy, ChangeSecretRecordStatusChoice
)
from accounts.models import (
Account, ChangeSecretAutomation,
@ -21,6 +22,7 @@ logger = get_logger(__file__)
__all__ = [
'ChangeSecretAutomationSerializer',
'ChangeSecretRecordSerializer',
'ChangeSecretRecordViewSecretSerializer',
'ChangeSecretRecordBackUpSerializer',
'ChangeSecretUpdateAssetSerializer',
'ChangeSecretUpdateNodeSerializer',
@ -104,7 +106,10 @@ class ChangeSecretAutomationSerializer(AuthValidateMixin, BaseAutomationSerializ
class ChangeSecretRecordSerializer(serializers.ModelSerializer):
is_success = serializers.SerializerMethodField(label=_('Is success'))
asset = ObjectRelatedField(queryset=Asset.objects, label=_('Asset'))
account = ObjectRelatedField(queryset=Account.objects, label=_('Account'))
account = ObjectRelatedField(
queryset=Account.objects, label=_('Account'),
attrs=("id", "name", "username")
)
execution = ObjectRelatedField(
queryset=AutomationExecution.objects, label=_('Automation task execution')
)
@ -119,7 +124,16 @@ class ChangeSecretRecordSerializer(serializers.ModelSerializer):
@staticmethod
def get_is_success(obj):
return obj.status == 'success'
return obj.status == ChangeSecretRecordStatusChoice.success.value
class ChangeSecretRecordViewSecretSerializer(serializers.ModelSerializer):
class Meta:
model = ChangeSecretRecord
fields = [
'id', 'old_secret', 'new_secret',
]
read_only_fields = fields
class ChangeSecretRecordBackUpSerializer(serializers.ModelSerializer):
@ -145,7 +159,7 @@ class ChangeSecretRecordBackUpSerializer(serializers.ModelSerializer):
@staticmethod
def get_is_success(obj):
if obj.status == 'success':
if obj.status == ChangeSecretRecordStatusChoice.success.value:
return _("Success")
return _("Failed")

View File

@ -36,14 +36,14 @@ def execute_account_automation_task(pid, trigger, tp):
instance.execute(trigger)
def record_task_activity_callback(self, record_id, *args, **kwargs):
def record_task_activity_callback(self, record_ids, *args, **kwargs):
from accounts.models import ChangeSecretRecord
with tmp_to_root_org():
record = get_object_or_none(ChangeSecretRecord, id=record_id)
if not record:
records = ChangeSecretRecord.objects.filter(id__in=record_ids)
if not records:
return
resource_ids = [record.id]
org_id = record.execution.org_id
resource_ids = [str(i.id) for i in records]
org_id = records[0].execution.org_id
return resource_ids, org_id
@ -51,22 +51,26 @@ def record_task_activity_callback(self, record_id, *args, **kwargs):
queue='ansible', verbose_name=_('Execute automation record'),
activity_callback=record_task_activity_callback
)
def execute_automation_record_task(record_id, tp):
def execute_automation_record_task(record_ids, tp):
from accounts.models import ChangeSecretRecord
task_name = gettext_noop('Execute automation record')
with tmp_to_root_org():
instance = get_object_or_none(ChangeSecretRecord, pk=record_id)
if not instance:
logger.error("No automation record found: {}".format(record_id))
records = ChangeSecretRecord.objects.filter(id__in=record_ids)
if not records:
logger.error('No automation record found: {}'.format(record_ids))
return
task_name = gettext_noop('Execute automation record')
record = records[0]
record_map = {f'{record.asset_id}-{record.account_id}': str(record.id) for record in records}
task_snapshot = {
'secret': instance.new_secret,
'secret_type': instance.execution.snapshot.get('secret_type'),
'accounts': [str(instance.account_id)],
'assets': [str(instance.asset_id)],
'params': {},
'record_id': record_id,
'record_map': record_map,
'secret': record.new_secret,
'secret_type': record.execution.snapshot.get('secret_type'),
'assets': [str(instance.asset_id) for instance in records],
'accounts': [str(instance.account_id) for instance in records],
}
with tmp_to_org(instance.execution.org_id):
with tmp_to_org(record.execution.org_id):
quickstart_automation_by_snapshot(task_name, tp, task_snapshot)

View File

@ -55,7 +55,7 @@ def clean_historical_accounts():
history_model = Account.history.model
history_id_mapper = defaultdict(list)
ids = history_model.objects.values('id').annotate(count=Count('id', distinct=True)) \
ids = history_model.objects.values('id').annotate(count=Count('id')) \
.filter(count__gte=limit).values_list('id', flat=True)
if not ids:

View File

@ -29,7 +29,8 @@ def template_sync_related_accounts(template_id, user_id=None):
name = template.name
username = template.username
secret_type = template.secret_type
print(f'\033[32m>>> 开始同步模版名称、用户名、密钥类型到相关联的账号 ({datetime.now().strftime("%Y-%m-%d %H:%M:%S")})')
print(
f'\033[32m>>> 开始同步模板名称、用户名、密钥类型到相关联的账号 ({datetime.now().strftime("%Y-%m-%d %H:%M:%S")})')
with tmp_to_org(org_id):
for account in accounts:
account.name = name

View File

@ -1,10 +1,10 @@
{% load i18n %}
<h3>{% trans 'Gather account change information' %}</h3>
<table style="width: 100%; border-collapse: collapse; max-width: 100%; text-align: left; margin-top: 20px;">
<caption></caption>
<tr style="background-color: #f2f2f2;">
<th style="border: 1px solid #ddd; padding: 10px; font-weight: bold;">{% trans 'Asset' %}</th>
<th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Asset' %}</th>
<th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Added account' %}</th>
<th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Deleted account' %}</th>
</tr>

View File

@ -0,0 +1,36 @@
{% load i18n %}
<h3>{% trans 'Task name' %}: {{ name }}</h3>
<h3>{% trans 'Task execution id' %}: {{ execution_id }}</h3>
<p>{% trans 'Respectful' %} {{ recipient }}</p>
<p>{% trans 'Hello! The following is the failure of changing the password of your assets or pushing the account. Please check and handle it in time.' %}</p>
<table style="width: 100%; border-collapse: collapse; max-width: 100%; text-align: left; margin-top: 20px;">
<caption></caption>
<thead>
<tr style="background-color: #f2f2f2;">
<th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Asset' %}</th>
<th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Account' %}</th>
<th style="border: 1px solid #ddd; padding: 10px;">{% trans 'Error' %}</th>
</tr>
</thead>
<tbody>
{% for asset_name, account_username, error in asset_account_errors %}
<tr>
<td style="border: 1px solid #ddd; padding: 10px;">{{ asset_name }}</td>
<td style="border: 1px solid #ddd; padding: 10px;">{{ account_username }}</td>
<td style="border: 1px solid #ddd; padding: 10px;">
<div style="
max-width: 90%;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
display: block;"
title="{{ error }}"
>
{{ error }}
</div>
</td>
</tr>
{% endfor %}
</tbody>
</table>

View File

@ -41,21 +41,21 @@ class UserLoginReminderMsg(UserMessage):
class AssetLoginReminderMsg(UserMessage):
subject = _('Asset login reminder')
def __init__(self, user, asset: Asset, login_user: User, account_username):
def __init__(self, user, asset: Asset, login_user: User, account: Account, input_username):
self.asset = asset
self.login_user = login_user
self.account_username = account_username
self.account = account
self.input_username = input_username
super().__init__(user)
def get_html_msg(self) -> dict:
account = Account.objects.get(asset=self.asset, username=self.account_username)
context = {
'recipient': self.user,
'username': self.login_user.username,
'name': self.login_user.name,
'asset': str(self.asset),
'account': self.account_username,
'account_name': account.name,
'account': self.input_username,
'account_name': self.account.name,
}
message = render_to_string('acls/asset_login_reminder.html', context)

View File

@ -32,6 +32,7 @@ __all__ = [
class AssetFilterSet(BaseFilterSet):
platform = django_filters.CharFilter(method='filter_platform')
exclude_platform = django_filters.CharFilter(field_name="platform__name", lookup_expr='exact', exclude=True)
domain = django_filters.CharFilter(method='filter_domain')
type = django_filters.CharFilter(field_name="platform__type", lookup_expr="exact")
category = django_filters.CharFilter(field_name="platform__category", lookup_expr="exact")
@ -92,7 +93,6 @@ class AssetViewSet(SuggestionMixin, OrgBulkModelViewSet):
model = Asset
filterset_class = AssetFilterSet
search_fields = ("name", "address", "comment")
ordering = ('name',)
ordering_fields = ('name', 'address', 'connectivity', 'platform', 'date_updated', 'date_created')
serializer_classes = (
("default", serializers.AssetSerializer),

View File

@ -48,7 +48,7 @@ class AssetPermUserListApi(BaseAssetPermUserOrUserGroupListApi):
def get_queryset(self):
perms = self.get_asset_related_perms()
users = User.objects.filter(
users = User.get_queryset().filter(
Q(assetpermissions__in=perms) | Q(groups__assetpermissions__in=perms)
).distinct()
return users

View File

@ -19,7 +19,6 @@ class DomainViewSet(OrgBulkModelViewSet):
model = Domain
filterset_fields = ("name",)
search_fields = filterset_fields
ordering = ('name',)
serializer_classes = {
'default': serializers.DomainSerializer,
'list': serializers.DomainListSerializer,
@ -30,6 +29,10 @@ class DomainViewSet(OrgBulkModelViewSet):
return serializers.DomainWithGatewaySerializer
return super().get_serializer_class()
def partial_update(self, request, *args, **kwargs):
kwargs['partial'] = True
return self.update(request, *args, **kwargs)
class GatewayViewSet(HostViewSet):
perm_model = Gateway

View File

@ -22,6 +22,7 @@ from orgs.utils import current_org
from rbac.permissions import RBACPermission
from .. import serializers
from ..models import Node
from ..signal_handlers import update_nodes_assets_amount
from ..tasks import (
update_node_assets_hardware_info_manual,
test_node_assets_connectivity_manual,
@ -94,6 +95,7 @@ class NodeAddChildrenApi(generics.UpdateAPIView):
children = Node.objects.filter(id__in=node_ids)
for node in children:
node.parent = instance
update_nodes_assets_amount.delay(ttl=5, node_ids=(instance.id,))
return Response("OK")

View File

@ -21,6 +21,7 @@ class AssetPlatformViewSet(JMSModelViewSet):
}
filterset_fields = ['name', 'category', 'type']
search_fields = ['name']
ordering = ['-internal', 'name']
rbac_perms = {
'categories': 'assets.view_platform',
'type_constraints': 'assets.view_platform',

View File

@ -1,2 +1,2 @@
from .endpoint import ExecutionManager
from .methods import platform_automation_methods, filter_platform_methods
from .methods import platform_automation_methods, filter_platform_methods, sorted_methods

View File

@ -12,7 +12,8 @@ from sshtunnel import SSHTunnelForwarder
from assets.automations.methods import platform_automation_methods
from common.utils import get_logger, lazyproperty, is_openssh_format_key, ssh_pubkey_gen
from ops.ansible import JMSInventory, PlaybookRunner, DefaultCallback
from ops.ansible import JMSInventory, DefaultCallback, SuperPlaybookRunner
from ops.ansible.interface import interface
logger = get_logger(__name__)
@ -54,7 +55,9 @@ class SSHTunnelManager:
not_valid.append(k)
else:
local_bind_port = server.local_bind_port
host['ansible_host'] = jms_asset['address'] = host['login_host'] = '127.0.0.1'
host['ansible_host'] = jms_asset['address'] = host[
'login_host'] = interface.get_gateway_proxy_host()
host['ansible_port'] = jms_asset['port'] = host['login_port'] = local_bind_port
servers.append(server)
@ -269,7 +272,7 @@ class BasePlaybookManager:
if not playbook_path:
continue
runer = PlaybookRunner(
runer = SuperPlaybookRunner(
inventory_path,
playbook_path,
self.runtime_dir,
@ -314,7 +317,7 @@ class BasePlaybookManager:
def delete_runtime_dir(self):
if settings.DEBUG_DEV:
return
shutil.rmtree(self.runtime_dir)
shutil.rmtree(self.runtime_dir, ignore_errors=True)
def run(self, *args, **kwargs):
print(">>> 任务准备阶段\n")
@ -333,6 +336,7 @@ class BasePlaybookManager:
ssh_tunnel = SSHTunnelManager()
ssh_tunnel.local_gateway_prepare(runner)
try:
kwargs.update({"clean_workspace": False})
cb = runner.run(**kwargs)
self.on_runner_success(runner, cb)
except Exception as e:

View File

@ -68,6 +68,10 @@ def filter_platform_methods(category, tp_name, method=None, methods=None):
return methods
def sorted_methods(methods):
return sorted(methods, key=lambda x: x.get('priority', 10))
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
platform_automation_methods = get_platform_automation_methods(BASE_DIR)

View File

@ -3,6 +3,7 @@
vars:
ansible_shell_type: sh
ansible_connection: local
ansible_python_interpreter: /opt/py3/bin/python
tasks:
- name: Test asset connection (pyfreerdp)

View File

@ -7,6 +7,7 @@ type:
- windows
method: ping
protocol: rdp
priority: 1
i18n:
Ping by pyfreerdp:

View File

@ -19,3 +19,6 @@
become_user: "{{ custom_become_user | default('') }}"
become_password: "{{ custom_become_password | default('') }}"
become_private_key_path: "{{ custom_become_private_key_path | default(None) }}"
old_ssh_version: "{{ jms_asset.old_ssh_version | default(False) }}"
gateway_args: "{{ jms_asset.ansible_ssh_common_args | default(None) }}"

View File

@ -7,6 +7,7 @@ type:
- all
method: ping
protocol: ssh
priority: 50
i18n:
Ping by paramiko:

View File

@ -0,0 +1,11 @@
- hosts: custom
gather_facts: no
vars:
ansible_connection: local
ansible_shell_type: sh
tasks:
- name: Test asset connection (telnet)
telnet_ping:
login_host: "{{ jms_asset.address }}"
login_port: "{{ jms_asset.port }}"

View File

@ -0,0 +1,16 @@
id: ping_by_telnet
name: "{{ 'Ping by telnet' | trans }}"
category:
- device
- host
type:
- all
method: ping
protocol: telnet
priority: 50
i18n:
Ping by telnet:
zh: '使用 Python 模块 telnet 测试主机可连接性'
en: 'Ping by telnet module'
ja: 'Pythonモジュールtelnetを使用したホスト接続性のテスト'

View File

@ -25,14 +25,22 @@ class PingManager(BasePlaybookManager):
def on_host_success(self, host, result):
asset, account = self.host_asset_and_account_mapper.get(host)
asset.set_connectivity(Connectivity.OK)
if not account:
return
account.set_connectivity(Connectivity.OK)
try:
asset.set_connectivity(Connectivity.OK)
if not account:
return
account.set_connectivity(Connectivity.OK)
except Exception as e:
print(f'\033[31m Update account {account.name} or '
f'update asset {asset.name} connectivity failed: {e} \033[0m\n')
def on_host_error(self, host, error, result):
asset, account = self.host_asset_and_account_mapper.get(host)
asset.set_connectivity(Connectivity.ERR)
if not account:
return
account.set_connectivity(Connectivity.ERR)
try:
asset.set_connectivity(Connectivity.ERR)
if not account:
return
account.set_connectivity(Connectivity.ERR)
except Exception as e:
print(f'\033[31m Update account {account.name} or '
f'update asset {asset.name} connectivity failed: {e} \033[0m\n')

View File

@ -92,18 +92,26 @@ class PingGatewayManager:
@staticmethod
def on_host_success(gateway, account):
print('\033[32m {} -> {}\033[0m\n'.format(gateway, account))
gateway.set_connectivity(Connectivity.OK)
if not account:
return
account.set_connectivity(Connectivity.OK)
try:
gateway.set_connectivity(Connectivity.OK)
if not account:
return
account.set_connectivity(Connectivity.OK)
except Exception as e:
print(f'\033[31m Update account {account.name} or '
f'update asset {gateway.name} connectivity failed: {e} \033[0m\n')
@staticmethod
def on_host_error(gateway, account, error):
print('\033[31m {} -> {} 原因: {} \033[0m\n'.format(gateway, account, error))
gateway.set_connectivity(Connectivity.ERR)
if not account:
return
account.set_connectivity(Connectivity.ERR)
try:
gateway.set_connectivity(Connectivity.ERR)
if not account:
return
account.set_connectivity(Connectivity.ERR)
except Exception as e:
print(f'\033[31m Update account {account.name} or '
f'update asset {gateway.name} connectivity failed: {e} \033[0m\n')
@staticmethod
def before_runner_start():

View File

@ -38,6 +38,14 @@ class Protocol(ChoicesMixin, models.TextChoices):
cls.ssh: {
'port': 22,
'secret_types': ['password', 'ssh_key'],
'setting': {
'old_ssh_version': {
'type': 'bool',
'default': False,
'label': _('Old SSH version'),
'help_text': _('Old SSH version like openssh 5.x or 6.x')
}
}
},
cls.sftp: {
'port': 22,
@ -187,6 +195,14 @@ class Protocol(ChoicesMixin, models.TextChoices):
'port': 27017,
'required': True,
'secret_types': ['password'],
'setting': {
'auth_source': {
'type': 'str',
'default': 'admin',
'label': _('Auth source'),
'help_text': _('The database to authenticate against')
}
}
},
cls.redis: {
'port': 6379,

View File

@ -90,7 +90,7 @@ class AllTypes(ChoicesMixin):
@classmethod
def set_automation_methods(cls, category, tp_name, constraints):
from assets.automations import filter_platform_methods
from assets.automations import filter_platform_methods, sorted_methods
automation = constraints.get('automation', {})
automation_methods = {}
platform_automation_methods = cls.get_automation_methods()
@ -101,6 +101,7 @@ class AllTypes(ChoicesMixin):
methods = filter_platform_methods(
category, tp_name, item_name, methods=platform_automation_methods
)
methods = sorted_methods(methods)
methods = [{'name': m['name'], 'id': m['id']} for m in methods]
automation_methods[item_name + '_methods'] = methods
automation.update(automation_methods)

View File

@ -1,6 +1,7 @@
# Generated by Django 3.2.12 on 2022-07-11 06:13
import time
import math
from django.utils import timezone
from itertools import groupby
from django.db import migrations
@ -40,9 +41,13 @@ def migrate_asset_accounts(apps, schema_editor):
if system_user:
# 更新一次系统用户的认证属性
account_values.update({attr: getattr(system_user, attr, '') for attr in all_attrs})
account_values['created_by'] = str(system_user.id)
account_values['privileged'] = system_user.type == 'admin' \
or system_user.username in ['root', 'Administrator']
if system_user.su_enabled and system_user.su_from:
created_by = f'{str(system_user.id)}::{str(system_user.su_from.username)}'
else:
created_by = str(system_user.id)
account_values['created_by'] = created_by
auth_book_auth = {attr: getattr(auth_book, attr, '') for attr in all_attrs if getattr(auth_book, attr, '')}
# 最终优先使用 auth_book 的认证属性
@ -117,6 +122,70 @@ def migrate_asset_accounts(apps, schema_editor):
print("\t - histories: {}".format(len(accounts_to_history)))
def update_asset_accounts_su_from(apps, schema_editor):
# Update accounts su_from
print("\n\tStart update asset accounts su_from field")
account_model = apps.get_model('accounts', 'Account')
platform_model = apps.get_model('assets', 'Platform')
asset_model = apps.get_model('assets', 'Asset')
platform_ids = list(platform_model.objects.filter(su_enabled=True).values_list('id', flat=True))
count = 0
step_size = 1000
count_account = 0
while True:
start = time.time()
asset_ids = asset_model.objects \
.filter(platform_id__in=platform_ids) \
.values_list('id', flat=True)[count:count + step_size]
asset_ids = list(asset_ids)
if not asset_ids:
break
count += len(asset_ids)
accounts = list(account_model.objects.filter(asset_id__in=asset_ids))
# {asset_id_account_username: account.id}}
asset_accounts_mapper = {}
for a in accounts:
try:
k = f'{a.asset_id}_{a.username}'
asset_accounts_mapper[k] = str(a.id)
except Exception as e:
pass
update_accounts = []
for a in accounts:
try:
if not a.created_by:
continue
created_by_list = a.created_by.split('::')
if len(created_by_list) != 2:
continue
su_from_username = created_by_list[1]
if not su_from_username:
continue
k = f'{a.asset_id}_{su_from_username}'
su_from_id = asset_accounts_mapper.get(k)
if not su_from_id:
continue
a.su_from_id = su_from_id
update_accounts.append(a)
except Exception as e:
pass
count_account += len(update_accounts)
log_msg = "\t - [{}]: Update accounts su_from: {}-{} {:.2f}s"
try:
account_model.objects.bulk_update(update_accounts, ['su_from_id'])
except Exception as e:
status = 'Failed'
else:
status = 'Success'
print(log_msg.format(status, count_account - len(update_accounts), count_account, time.time() - start))
def migrate_db_accounts(apps, schema_editor):
app_perm_model = apps.get_model('perms', 'ApplicationPermission')
account_model = apps.get_model('accounts', 'Account')
@ -196,5 +265,6 @@ class Migration(migrations.Migration):
operations = [
migrations.RunPython(migrate_asset_accounts),
migrations.RunPython(update_asset_accounts_su_from),
migrations.RunPython(migrate_db_accounts),
]

View File

@ -73,3 +73,7 @@ class Gateway(Host):
def private_key_path(self):
account = self.select_account
return account.private_key_path if account else None
def get_private_key_path(self, path):
account = self.select_account
return account.get_private_key_path(path) if account else None

View File

@ -73,6 +73,10 @@ class FamilyMixin:
@classmethod
def get_nodes_all_children(cls, nodes, with_self=True):
pattern = cls.get_nodes_children_key_pattern(nodes, with_self=with_self)
if not pattern:
# 如果 pattern = ''
# key__iregex 报错 (1139, "Got error 'empty (sub)expression' from regexp")
return cls.objects.none()
return Node.objects.filter(key__iregex=pattern)
@classmethod

View File

@ -1,8 +1,7 @@
from django.db import models
from django.utils.translation import gettext_lazy as _
from assets.const import AllTypes
from assets.const import Protocol
from assets.const import AllTypes, Category, Protocol
from common.db.fields import JsonDictTextField
from common.db.models import JMSBaseModel
@ -119,6 +118,15 @@ class Platform(LabeledMixin, JMSBaseModel):
)
return linux.id
def is_huawei(self):
if self.category != Category.DEVICE:
return False
if 'huawei' in self.name.lower():
return True
if '华为' in self.name:
return True
return False
def __str__(self):
return self.name

View File

@ -22,6 +22,36 @@ class WebSpecSerializer(serializers.ModelSerializer):
'submit_selector', 'script'
]
def get_fields(self):
fields = super().get_fields()
if self.is_retrieve():
# 查看 Web 资产详情时
self.pop_fields_if_need(fields)
return fields
def is_retrieve(self):
try:
self.context.get('request').method and self.parent.instance.web
return True
except Exception:
return False
def pop_fields_if_need(self, fields):
fields_script = ['script']
fields_basic = ['username_selector', 'password_selector', 'submit_selector']
autofill = self.parent.instance.web.autofill
pop_fields_mapper = {
FillType.no: fields_script + fields_basic,
FillType.basic: fields_script,
FillType.script: fields_basic,
}
fields_pop = pop_fields_mapper.get(autofill, [])
for f in fields_pop:
fields.pop(f, None)
return fields
category_spec_serializer_map = {
'database': DatabaseSpecSerializer,

View File

@ -1,12 +1,13 @@
# -*- coding: utf-8 -*-
#
from django.db.models import Count
from django.db.models import Count, Q
from django.utils.translation import gettext_lazy as _
from rest_framework import serializers
from common.serializers import ResourceLabelsMixin
from common.serializers.fields import ObjectRelatedField
from orgs.mixins.serializers import BulkOrgResourceModelSerializer
from assets.models.gateway import Gateway
from .gateway import GatewayWithAccountSecretSerializer
from ..models import Domain
@ -15,7 +16,7 @@ __all__ = ['DomainSerializer', 'DomainWithGatewaySerializer', 'DomainListSeriali
class DomainSerializer(ResourceLabelsMixin, BulkOrgResourceModelSerializer):
gateways = ObjectRelatedField(
many=True, required=False, label=_('Gateways'), read_only=True,
many=True, required=False, label=_('Gateway'), queryset=Gateway.objects
)
assets_amount = serializers.IntegerField(label=_('Assets amount'), read_only=True)
@ -27,7 +28,7 @@ class DomainSerializer(ResourceLabelsMixin, BulkOrgResourceModelSerializer):
read_only_fields = ['date_created']
fields = fields_small + fields_m2m + read_only_fields
extra_kwargs = {
'assets': {'label': _("Assets")}
'assets': {'required': False},
}
def to_representation(self, instance):
@ -39,12 +40,17 @@ class DomainSerializer(ResourceLabelsMixin, BulkOrgResourceModelSerializer):
data['assets'] = [i for i in assets if str(i['id']) not in gateway_ids]
return data
def update(self, instance, validated_data):
def create(self, validated_data):
assets = validated_data.pop('assets', [])
assets = assets + list(instance.gateways)
validated_data['assets'] = assets
instance = super().update(instance, validated_data)
return instance
gateways = validated_data.pop('gateways', [])
validated_data['assets'] = assets + gateways
return super().create(validated_data)
def update(self, instance, validated_data):
assets = validated_data.pop('assets', list(instance.assets.all()))
gateways = validated_data.pop('gateways', list(instance.gateways.all()))
validated_data['assets'] = assets + gateways
return super().update(instance, validated_data)
@classmethod
def setup_eager_loading(cls, queryset):
@ -61,7 +67,7 @@ class DomainListSerializer(DomainSerializer):
@classmethod
def setup_eager_loading(cls, queryset):
queryset = queryset.annotate(
assets_amount=Count('assets', distinct=True),
assets_amount=Count('assets', filter=~Q(assets__platform__name='Gateway'), distinct=True),
)
return queryset

View File

@ -88,8 +88,7 @@ class KubernetesClient:
try:
data = getattr(self, func_name)(*args)
except Exception as e:
logger.error(e)
raise e
logger.error(f'K8S tree get {tp} error: {e}')
if self.server:
self.server.stop()

View File

@ -1,6 +1,5 @@
# -*- coding: utf-8 -*-
#
from importlib import import_module
from django.conf import settings
@ -31,7 +30,7 @@ from terminal.models import default_storage
from users.models import User
from .backends import TYPE_ENGINE_MAPPING
from .const import ActivityChoices
from .filters import UserSessionFilterSet
from .filters import UserSessionFilterSet, OperateLogFilterSet
from .models import (
FTPLog, UserLoginLog, OperateLog, PasswordChangeLog,
ActivityLog, JobLog, UserSession
@ -66,7 +65,7 @@ class FTPLogViewSet(OrgModelViewSet):
date_range_filter_fields = [
('date_start', ('date_from', 'date_to'))
]
filterset_fields = ['user', 'asset', 'account', 'filename']
filterset_fields = ['user', 'asset', 'account', 'filename', 'session']
search_fields = filterset_fields
ordering = ['-date_start']
http_method_names = ['post', 'get', 'head', 'options', 'patch']
@ -205,10 +204,7 @@ class OperateLogViewSet(OrgReadonlyModelViewSet):
date_range_filter_fields = [
('datetime', ('date_from', 'date_to'))
]
filterset_fields = [
'user', 'action', 'resource_type', 'resource',
'remote_addr'
]
filterset_class = OperateLogFilterSet
search_fields = ['resource', 'user']
ordering = ['-datetime']
@ -272,7 +268,7 @@ class UserSessionViewSet(CommonApiMixin, viewsets.ModelViewSet):
return user_ids
def get_queryset(self):
keys = UserSession.get_keys()
keys = user_session_manager.get_keys()
queryset = UserSession.objects.filter(key__in=keys)
if current_org.is_root():
return queryset
@ -291,6 +287,6 @@ class UserSessionViewSet(CommonApiMixin, viewsets.ModelViewSet):
keys = queryset.values_list('key', flat=True)
for key in keys:
user_session_manager.decrement_or_remove(key)
user_session_manager.remove(key)
queryset.delete()
return Response(status=status.HTTP_200_OK)

View File

@ -1,11 +1,13 @@
from django.apps import apps
from django.utils import translation
from django_filters import rest_framework as drf_filters
from rest_framework import filters
from rest_framework.compat import coreapi, coreschema
from common.drf.filters import BaseFilterSet
from common.sessions.cache import user_session_manager
from orgs.utils import current_org
from .models import UserSession
from .models import UserSession, OperateLog
__all__ = ['CurrentOrgMembersFilter']
@ -50,3 +52,22 @@ class UserSessionFilterSet(BaseFilterSet):
class Meta:
model = UserSession
fields = ['id', 'ip', 'city', 'type']
class OperateLogFilterSet(BaseFilterSet):
resource_type = drf_filters.CharFilter(method='filter_resource_type')
@staticmethod
def filter_resource_type(queryset, name, resource_type):
current_lang = translation.get_language()
with translation.override(current_lang):
mapper = {str(m._meta.verbose_name): m._meta.verbose_name_raw for m in apps.get_models()}
tp = mapper.get(resource_type)
queryset = queryset.filter(resource_type=tp)
return queryset
class Meta:
model = OperateLog
fields = [
'user', 'action', 'resource', 'remote_addr'
]

View File

@ -12,7 +12,10 @@ from common.utils.timezone import as_current_tz
from jumpserver.utils import current_request
from orgs.models import Organization
from orgs.utils import get_current_org_id
from settings.models import Setting
from settings.serializers import SettingsSerializer
from users.models import Preference
from users.serializers import PreferenceSerializer
from .backends import get_operate_log_storage
logger = get_logger(__name__)
@ -87,19 +90,15 @@ class OperatorLogHandler(metaclass=Singleton):
return log_id, before, after
@staticmethod
def get_resource_display_from_setting(resource):
resource_display = None
setting_serializer = SettingsSerializer()
label = setting_serializer.get_field_label(resource)
if label is not None:
resource_display = label
return resource_display
def get_resource_display(self, resource):
resource_display = str(resource)
return_value = self.get_resource_display_from_setting(resource_display)
if return_value is not None:
resource_display = return_value
def get_resource_display(resource):
if isinstance(resource, Setting):
serializer = SettingsSerializer()
resource_display = serializer.get_field_label(resource.name)
elif isinstance(resource, Preference):
serializer = PreferenceSerializer()
resource_display = serializer.get_field_label(resource.name)
else:
resource_display = str(resource)
return resource_display
@staticmethod

View File

@ -288,16 +288,9 @@ class UserSession(models.Model):
ttl = caches[settings.SESSION_CACHE_ALIAS].ttl(cache_key)
return timezone.now() + timedelta(seconds=ttl)
@staticmethod
def get_keys():
session_store_cls = import_module(settings.SESSION_ENGINE).SessionStore
cache_key_prefix = session_store_cls.cache_key_prefix
keys = caches[settings.SESSION_CACHE_ALIAS].iter_keys('*')
return [k.replace(cache_key_prefix, '') for k in keys]
@classmethod
def clear_expired_sessions(cls):
keys = cls.get_keys()
keys = user_session_manager.get_keys()
cls.objects.exclude(key__in=keys).delete()
class Meta:

View File

@ -23,7 +23,7 @@ class JobLogSerializer(JobExecutionSerializer):
class Meta:
model = models.JobLog
read_only_fields = [
"id", "material", "time_cost", 'date_start',
"id", "material", 'job_type', "time_cost", 'date_start',
'date_finished', 'date_created',
'is_finished', 'is_success',
'task_id', 'creator_name'
@ -43,7 +43,7 @@ class FTPLogSerializer(serializers.ModelSerializer):
fields_small = fields_mini + [
"user", "remote_addr", "asset", "account",
"org_id", "operate", "filename", "date_start",
"is_success", "has_file",
"is_success", "has_file", "session"
]
fields = fields_small

View File

@ -36,6 +36,7 @@ class AuthBackendLabelMapping(LazyObject):
backend_label_mapping[settings.AUTH_BACKEND_AUTH_TOKEN] = _("Auth Token")
backend_label_mapping[settings.AUTH_BACKEND_WECOM] = _("WeCom")
backend_label_mapping[settings.AUTH_BACKEND_FEISHU] = _("FeiShu")
backend_label_mapping[settings.AUTH_BACKEND_LARK] = 'Lark'
backend_label_mapping[settings.AUTH_BACKEND_SLACK] = _("Slack")
backend_label_mapping[settings.AUTH_BACKEND_DINGTALK] = _("DingTalk")
backend_label_mapping[settings.AUTH_BACKEND_TEMP_TOKEN] = _("Temporary token")

View File

@ -178,7 +178,7 @@ def on_django_start_set_operate_log_monitor_models(sender, **kwargs):
'PermedAsset', 'PermedAccount', 'MenuPermission',
'Permission', 'TicketSession', 'ApplyLoginTicket',
'ApplyCommandTicket', 'ApplyLoginAssetTicket',
'FavoriteAsset', 'Asset'
'FavoriteAsset',
}
for i, app in enumerate(apps.get_models(), 1):
app_name = app._meta.app_label

View File

@ -7,19 +7,18 @@ import subprocess
from celery import shared_task
from django.conf import settings
from django.core.files.storage import default_storage
from django.db import transaction
from django.utils import timezone
from django.utils.translation import gettext_lazy as _
from common.const.crontab import CRONTAB_AT_AM_TWO
from common.utils import get_log_keep_day, get_logger
from common.storage.ftp_file import FTPFileStorageHandler
from ops.celery.decorator import (
register_as_period_task, after_app_shutdown_clean_periodic
)
from common.utils import get_log_keep_day, get_logger
from ops.celery.decorator import register_as_period_task
from ops.models import CeleryTaskExecution
from terminal.models import Session, Command
from terminal.backends import server_replay_storage
from .models import UserLoginLog, OperateLog, FTPLog, ActivityLog
from terminal.models import Session, Command
from .models import UserLoginLog, OperateLog, FTPLog, ActivityLog, PasswordChangeLog
logger = get_logger(__name__)
@ -38,6 +37,14 @@ def clean_operation_log_period():
OperateLog.objects.filter(datetime__lt=expired_day).delete()
def clean_password_change_log_period():
now = timezone.now()
days = get_log_keep_day('PASSWORD_CHANGE_LOG_KEEP_DAYS')
expired_day = now - datetime.timedelta(days=days)
PasswordChangeLog.objects.filter(datetime__lt=expired_day).delete()
logger.info("Clean password change log done")
def clean_activity_log_period():
now = timezone.now()
days = get_log_keep_day('ACTIVITY_LOG_KEEP_DAYS')
@ -49,9 +56,9 @@ def clean_ftp_log_period():
now = timezone.now()
days = get_log_keep_day('FTP_LOG_KEEP_DAYS')
expired_day = now - datetime.timedelta(days=days)
file_store_dir = os.path.join(default_storage.base_location, 'ftp_file')
file_store_dir = os.path.join(default_storage.base_location, FTPLog.upload_to)
FTPLog.objects.filter(date_start__lt=expired_day).delete()
command = "find %s -mtime +%s -exec rm -f {} \\;" % (
command = "find %s -mtime +%s -type f -exec rm -f {} \\;" % (
file_store_dir, days
)
subprocess.call(command, shell=True)
@ -76,6 +83,15 @@ def clean_celery_tasks_period():
subprocess.call(command, shell=True)
def batch_delete(queryset, batch_size=3000):
model = queryset.model
count = queryset.count()
with transaction.atomic():
for i in range(0, count, batch_size):
pks = queryset[i:i + batch_size].values_list('id', flat=True)
model.objects.filter(id__in=list(pks)).delete()
def clean_expired_session_period():
logger.info("Start clean expired session record, commands and replay")
days = get_log_keep_day('TERMINAL_SESSION_KEEP_DURATION')
@ -85,9 +101,9 @@ def clean_expired_session_period():
expired_commands = Command.objects.filter(timestamp__lt=timestamp)
replay_dir = os.path.join(default_storage.base_location, 'replay')
expired_sessions.delete()
batch_delete(expired_sessions)
logger.info("Clean session item done")
expired_commands.delete()
batch_delete(expired_commands)
logger.info("Clean session command done")
command = "find %s -mtime +%s \\( -name '*.json' -o -name '*.tar' -o -name '*.gz' \\) -exec rm -f {} \\;" % (
replay_dir, days
@ -100,7 +116,6 @@ def clean_expired_session_period():
@shared_task(verbose_name=_('Clean audits session task log'))
@register_as_period_task(crontab=CRONTAB_AT_AM_TWO)
@after_app_shutdown_clean_periodic
def clean_audits_log_period():
print("Start clean audit session task log")
clean_login_log_period()
@ -109,6 +124,7 @@ def clean_audits_log_period():
clean_activity_log_period()
clean_celery_tasks_period()
clean_expired_session_period()
clean_password_change_log_period()
@shared_task(verbose_name=_('Upload FTP file to external storage'))

View File

@ -2,13 +2,15 @@
#
from .access_key import *
from .common import *
from .confirm import *
from .connection_token import *
from .feishu import *
from .lark import *
from .login_confirm import *
from .mfa import *
from .password import *
from .session import *
from .sso import *
from .temp_token import *
from .token import *
from .common import *

View File

@ -12,7 +12,6 @@ from common.permissions import IsValidUser, OnlySuperUser
from common.utils import get_logger
from users.models import User
logger = get_logger(__file__)
@ -24,6 +23,7 @@ class QRUnBindBase(APIView):
'wecom': {'user_field': 'wecom_id', 'not_bind_err': errors.WeComNotBound},
'dingtalk': {'user_field': 'dingtalk_id', 'not_bind_err': errors.DingTalkNotBound},
'feishu': {'user_field': 'feishu_id', 'not_bind_err': errors.FeiShuNotBound},
'lark': {'user_field': 'lark_id', 'not_bind_err': errors.LarkNotBound},
'slack': {'user_field': 'slack_id', 'not_bind_err': errors.SlackNotBound},
}
user = self.user

View File

@ -223,12 +223,17 @@ class ExtraActionApiMixin(RDPFileClientProtocolURLMixin):
validate_exchange_token: callable
@action(methods=['POST', 'GET'], detail=True, url_path='rdp-file')
def get_rdp_file(self, *args, **kwargs):
def get_rdp_file(self, request, *args, **kwargs):
token = self.get_object()
token.is_valid()
filename, content = self.get_rdp_file_info(token)
filename = '{}.rdp'.format(filename)
response = HttpResponse(content, content_type='application/octet-stream')
if is_true(request.query_params.get('reusable')):
token.set_reusable(True)
filename = '{}-{}'.format(filename, token.date_expired.strftime('%Y%m%d_%H%M%S'))
filename += '.rdp'
response['Content-Disposition'] = 'attachment; filename*=UTF-8\'\'%s' % filename
return response
@ -379,6 +384,7 @@ class ConnectionTokenViewSet(ExtraActionApiMixin, RootOrgViewMixin, JMSModelView
if account.username != AliasAccount.INPUT:
data['input_username'] = ''
ticket = self._validate_acl(user, asset, account)
if ticket:
data['from_ticket'] = ticket
@ -413,7 +419,10 @@ class ConnectionTokenViewSet(ExtraActionApiMixin, RootOrgViewMixin, JMSModelView
def _validate_acl(self, user, asset, account):
from acls.models import LoginAssetACL
acls = LoginAssetACL.filter_queryset(user=user, asset=asset, account=account)
kwargs = {'user': user, 'asset': asset, 'account': account}
if account.username == AliasAccount.INPUT:
kwargs['account_username'] = self.input_username
acls = LoginAssetACL.filter_queryset(**kwargs)
ip = get_request_ip_or_data(self.request)
acl = LoginAssetACL.get_match_rule_acls(user, ip, acls)
if not acl:
@ -443,7 +452,7 @@ class ConnectionTokenViewSet(ExtraActionApiMixin, RootOrgViewMixin, JMSModelView
self._record_operate_log(acl, asset)
for reviewer in reviewers:
AssetLoginReminderMsg(
reviewer, asset, user, self.input_username
reviewer, asset, user, account, self.input_username
).publish_async()
def create(self, request, *args, **kwargs):
@ -503,20 +512,16 @@ class SuperConnectionTokenViewSet(ConnectionTokenViewSet):
token.is_valid()
serializer = self.get_serializer(instance=token)
expire_now = request.data.get('expire_now', None)
expire_now = request.data.get('expire_now', True)
asset_type = token.asset.type
# 设置默认值
if expire_now is None:
# TODO 暂时特殊处理 k8s 不过期
if asset_type in ['k8s', 'kubernetes']:
expire_now = False
else:
expire_now = not settings.CONNECTION_TOKEN_REUSABLE
if asset_type in ['k8s', 'kubernetes']:
expire_now = False
if is_false(expire_now):
logger.debug('Api specified, now expire now')
elif token.is_reusable and settings.CONNECTION_TOKEN_REUSABLE:
if token.is_reusable and settings.CONNECTION_TOKEN_REUSABLE:
logger.debug('Token is reusable, not expire now')
elif is_false(expire_now):
logger.debug('Api specified, now expire now')
else:
token.expire()

View File

@ -0,0 +1,8 @@
from common.utils import get_logger
from .feishu import FeiShuEventSubscriptionCallback
logger = get_logger(__name__)
class LarkEventSubscriptionCallback(FeiShuEventSubscriptionCallback):
pass

View File

@ -9,6 +9,7 @@ from common.utils import get_logger
from .. import errors, mixins
__all__ = ['TicketStatusApi']
logger = get_logger(__name__)

View File

@ -9,7 +9,7 @@ from rest_framework.permissions import AllowAny
from rest_framework.response import Response
from rest_framework.serializers import ValidationError
from common.exceptions import UnexpectError
from common.exceptions import JMSException, UnexpectError
from common.utils import get_logger
from users.models.user import User
from .. import errors
@ -50,7 +50,10 @@ class MFASendCodeApi(AuthMixin, CreateAPIView):
mfa_type = serializer.validated_data['type']
if not username:
user = self.get_user_from_session()
try:
user = self.get_user_from_session()
except errors.SessionEmptyError as e:
raise ValidationError({'error': e})
else:
user = self.get_user_from_db(username)
@ -61,6 +64,8 @@ class MFASendCodeApi(AuthMixin, CreateAPIView):
try:
mfa_backend.send_challenge()
except JMSException:
raise
except Exception as e:
raise UnexpectError(str(e))

View File

@ -0,0 +1,68 @@
import time
from threading import Thread
from django.conf import settings
from django.contrib.auth import logout
from django.contrib.auth.models import AnonymousUser
from rest_framework import generics
from rest_framework import status
from rest_framework.response import Response
from common.sessions.cache import user_session_manager
from common.utils import get_logger
__all__ = ['UserSessionApi']
logger = get_logger(__name__)
class UserSessionManager:
def __init__(self, request):
self.request = request
self.session = request.session
def connect(self):
user_session_manager.add_or_increment(self.session.session_key)
def disconnect(self):
user_session_manager.decrement(self.session.session_key)
if self.should_delete_session():
thread = Thread(target=self.delay_delete_session)
thread.start()
def should_delete_session(self):
return (self.session.modified or settings.SESSION_SAVE_EVERY_REQUEST) and \
not self.session.is_empty() and \
self.session.get_expire_at_browser_close() and \
not user_session_manager.check_active(self.session.session_key)
def delay_delete_session(self):
timeout = 6
check_interval = 0.5
start_time = time.time()
while time.time() - start_time < timeout:
time.sleep(check_interval)
if user_session_manager.check_active(self.session.session_key):
return
logout(self.request)
class UserSessionApi(generics.RetrieveDestroyAPIView):
permission_classes = ()
def retrieve(self, request, *args, **kwargs):
if isinstance(request.user, AnonymousUser):
return Response(status=status.HTTP_200_OK)
UserSessionManager(request).connect()
return Response(status=status.HTTP_200_OK)
def destroy(self, request, *args, **kwargs):
if isinstance(request.user, AnonymousUser):
return Response(status=status.HTTP_200_OK)
UserSessionManager(request).disconnect()
return Response(status=status.HTTP_204_NO_CONTENT)

View File

@ -5,11 +5,13 @@ from django.conf import settings
from django.contrib.auth import login
from django.http.response import HttpResponseRedirect
from rest_framework import serializers
from rest_framework import status
from rest_framework.decorators import action
from rest_framework.permissions import AllowAny
from rest_framework.request import Request
from rest_framework.response import Response
from authentication.errors import ACLError
from common.api import JMSGenericViewSet
from common.const.http import POST, GET
from common.permissions import OnlySuperUser
@ -17,7 +19,10 @@ from common.serializers import EmptySerializer
from common.utils import reverse, safe_next_url
from common.utils.timezone import utc_now
from users.models import User
from ..errors import SSOAuthClosed
from users.utils import LoginBlockUtil, LoginIpBlockUtil
from ..errors import (
SSOAuthClosed, AuthFailedError, LoginConfirmBaseError, SSOAuthKeyTTLError
)
from ..filters import AuthKeyQueryDeclaration
from ..mixins import AuthMixin
from ..models import SSOToken
@ -63,31 +68,58 @@ class SSOViewSet(AuthMixin, JMSGenericViewSet):
此接口违反了 `Restful` 的规范
`GET` 应该是安全的方法但此接口是不安全的
"""
status_code = status.HTTP_400_BAD_REQUEST
request.META['HTTP_X_JMS_LOGIN_TYPE'] = 'W'
authkey = request.query_params.get(AUTH_KEY)
next_url = request.query_params.get(NEXT_URL)
if not next_url or not next_url.startswith('/'):
next_url = reverse('index')
if not authkey:
raise serializers.ValidationError("authkey is required")
try:
if not authkey:
raise serializers.ValidationError("authkey is required")
authkey = UUID(authkey)
token = SSOToken.objects.get(authkey=authkey, expired=False)
# 先过期,只能访问这一次
except (ValueError, SSOToken.DoesNotExist, serializers.ValidationError) as e:
error_msg = str(e)
self.send_auth_signal(success=False, reason=error_msg)
return Response({'error': error_msg}, status=status_code)
error_msg = None
user = token.user
username = user.username
ip = self.get_request_ip()
try:
if (utc_now().timestamp() - token.date_created.timestamp()) > settings.AUTH_SSO_AUTHKEY_TTL:
raise SSOAuthKeyTTLError()
self._check_is_block(username, True)
self._check_only_allow_exists_user_auth(username)
self._check_login_acl(user, ip)
self.check_user_login_confirm_if_need(user)
self.request.session['auth_backend'] = settings.AUTH_BACKEND_SSO
login(self.request, user, settings.AUTH_BACKEND_SSO)
self.send_auth_signal(success=True, user=user)
self.mark_mfa_ok('otp', user)
LoginIpBlockUtil(ip).clean_block_if_need()
LoginBlockUtil(username, ip).clean_failed_count()
self.clear_auth_mark()
except (ACLError, LoginConfirmBaseError): # 无需记录日志
pass
except (AuthFailedError, SSOAuthKeyTTLError) as e:
error_msg = e.msg
except Exception as e:
error_msg = str(e)
finally:
token.expired = True
token.save()
except (ValueError, SSOToken.DoesNotExist):
self.send_auth_signal(success=False, reason='authkey_invalid')
return HttpResponseRedirect(next_url)
# 判断是否过期
if (utc_now().timestamp() - token.date_created.timestamp()) > settings.AUTH_SSO_AUTHKEY_TTL:
self.send_auth_signal(success=False, reason='authkey_timeout')
if error_msg:
self.send_auth_signal(success=False, username=username, reason=error_msg)
return Response({'error': error_msg}, status=status_code)
else:
return HttpResponseRedirect(next_url)
user = token.user
login(self.request, user, settings.AUTH_BACKEND_SSO)
self.send_auth_signal(success=True, user=user)
return HttpResponseRedirect(next_url)

View File

@ -4,10 +4,13 @@ from django.contrib import auth
from django.http import HttpResponseRedirect
from django.urls import reverse
from django.utils.http import urlencode
from django.utils.translation import gettext_lazy as _
from authentication.utils import build_absolute_uri
from common.utils import get_logger
from authentication.views.mixins import FlashMessageMixin
from authentication.mixins import authenticate
from common.utils import get_logger
logger = get_logger(__file__)
@ -39,7 +42,7 @@ class OAuth2AuthRequestView(View):
return HttpResponseRedirect(redirect_url)
class OAuth2AuthCallbackView(View):
class OAuth2AuthCallbackView(View, FlashMessageMixin):
http_method_names = ['get', ]
def get(self, request):
@ -51,6 +54,11 @@ class OAuth2AuthCallbackView(View):
if 'code' in callback_params:
logger.debug(log_prompt.format('Process authenticate'))
user = authenticate(code=callback_params['code'], request=request)
if err_msg := getattr(request, 'error_message', ''):
login_url = reverse('authentication:login') + '?admin=1'
return self.get_failed_response(login_url, title=_('Authentication failed'), msg=err_msg)
if user and user.is_valid:
logger.debug(log_prompt.format('Login: {}'.format(user)))
auth.login(self.request, user)

View File

@ -55,6 +55,12 @@ class FeiShuAuthentication(JMSModelBackend):
pass
class LarkAuthentication(FeiShuAuthentication):
@staticmethod
def is_enabled():
return settings.AUTH_LARK
class SlackAuthentication(JMSModelBackend):
"""
什么也不做呀😺
@ -72,5 +78,6 @@ class AuthorizationTokenAuthentication(JMSModelBackend):
"""
什么也不做呀😺
"""
def authenticate(self, request, **kwargs):
pass

View File

@ -52,6 +52,10 @@ class AuthFailedError(Exception):
return str(self.msg)
class SSOAuthKeyTTLError(Exception):
msg = 'sso_authkey_timeout'
class BlockGlobalIpLoginError(AuthFailedError):
error = 'block_global_ip_login'

View File

@ -33,6 +33,11 @@ class FeiShuNotBound(JMSException):
default_detail = _('FeiShu is not bound')
class LarkNotBound(JMSException):
default_code = 'lark_not_bound'
default_detail = _('Lark is not bound')
class SlackNotBound(JMSException):
default_code = 'slack_not_bound'
default_detail = _('Slack is not bound')

View File

@ -17,10 +17,6 @@ class EncryptedField(forms.CharField):
class UserLoginForm(forms.Form):
days_auto_login = int(settings.SESSION_COOKIE_AGE / 3600 / 24)
disable_days_auto_login = settings.SESSION_EXPIRE_AT_BROWSER_CLOSE \
or days_auto_login < 1
username = forms.CharField(
label=_('Username'), max_length=100,
widget=forms.TextInput(attrs={
@ -34,15 +30,15 @@ class UserLoginForm(forms.Form):
)
auto_login = forms.BooleanField(
required=False, initial=False,
widget=forms.CheckboxInput(
attrs={'disabled': disable_days_auto_login}
)
widget=forms.CheckboxInput()
)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
auto_login_field = self.fields['auto_login']
auto_login_field.label = _("{} days auto login").format(self.days_auto_login or 1)
auto_login_field.label = _("Auto login")
if settings.SESSION_EXPIRE_AT_BROWSER_CLOSE:
auto_login_field.widget = forms.HiddenInput()
def confirm_login_allowed(self, user):
if not user.is_staff:

View File

@ -363,7 +363,6 @@ class AuthACLMixin:
if acl.is_action(acl.ActionChoices.notice):
self.request.session['auth_notice_required'] = '1'
self.request.session['auth_acl_id'] = str(acl.id)
return
def _check_third_party_login_acl(self):
request = self.request

View File

@ -82,12 +82,15 @@ class ConnectionToken(JMSOrgBaseModel):
self.save(update_fields=['date_expired'])
def set_reusable(self, is_reusable):
if not settings.CONNECTION_TOKEN_REUSABLE:
return
self.is_reusable = is_reusable
if self.is_reusable:
seconds = settings.CONNECTION_TOKEN_REUSABLE_EXPIRATION
else:
seconds = settings.CONNECTION_TOKEN_ONETIME_EXPIRATION
self.date_expired = timezone.now() + timedelta(seconds=seconds)
self.date_expired = self.date_created + timedelta(seconds=seconds)
self.save(update_fields=['is_reusable', 'date_expired'])
def renewal(self):
@ -201,12 +204,14 @@ class ConnectionToken(JMSOrgBaseModel):
host, account, lock_key = bulk_get(host_account, ('host', 'account', 'lock_key'))
gateway = host.domain.select_gateway() if host.domain else None
platform = host.platform
data = {
'id': lock_key,
'applet': applet,
'host': host,
'gateway': gateway,
'platform': platform,
'account': account,
'remote_app_option': self.get_remote_app_option()
}

View File

@ -161,6 +161,7 @@ class ConnectTokenAppletOptionSerializer(serializers.Serializer):
host = _ConnectionTokenAssetSerializer(read_only=True)
account = _ConnectionTokenAccountSerializer(read_only=True)
gateway = _ConnectionTokenGatewaySerializer(read_only=True)
platform = _ConnectionTokenPlatformSerializer(read_only=True)
remote_app_option = serializers.JSONField(read_only=True)

View File

@ -1,5 +1,3 @@
from importlib import import_module
from django.conf import settings
from django.contrib.auth import user_logged_in
from django.core.cache import cache
@ -8,6 +6,7 @@ from django_cas_ng.signals import cas_user_authenticated
from apps.jumpserver.settings.auth import AUTHENTICATION_BACKENDS_THIRD_PARTY
from audits.models import UserSession
from common.sessions.cache import user_session_manager
from .signals import post_auth_success, post_auth_failed, user_auth_failed, user_auth_success
@ -32,8 +31,7 @@ def on_user_auth_login_success(sender, user, request, **kwargs):
lock_key = 'single_machine_login_' + str(user.id)
session_key = cache.get(lock_key)
if session_key and session_key != request.session.session_key:
session = import_module(settings.SESSION_ENGINE).SessionStore(session_key)
session.delete()
user_session_manager.remove(session_key)
UserSession.objects.filter(key=session_key).delete()
cache.set(lock_key, request.session.session_key, None)

View File

@ -406,6 +406,15 @@
$('#password-hidden').val(passwordEncrypted); //返回给密码输入input
$('#login-form').submit(); //post提交
}
function checkHealth() {
let url = "{% url 'health' %}";
requestApi({
url: url,
method: "GET",
flash_message: false,
})
}
setInterval(checkHealth, 30 * 1000);
</script>
</html>

View File

@ -95,6 +95,7 @@ function doRequestAuth() {
}
clearInterval(interval);
clearInterval(checkInterval);
cancelTicket();
$(".copy-btn").attr('disabled', 'disabled');
errorMsgRef.html(data.msg)
}

View File

@ -22,6 +22,9 @@ urlpatterns = [
path('feishu/event/subscription/callback/', api.FeiShuEventSubscriptionCallback.as_view(),
name='feishu-event-subscription-callback'),
path('lark/event/subscription/callback/', api.LarkEventSubscriptionCallback.as_view(),
name='lark-event-subscription-callback'),
path('auth/', api.TokenCreateApi.as_view(), name='user-auth'),
path('confirm-oauth/', api.ConfirmBindORUNBindOAuth.as_view(), name='confirm-oauth'),
path('tokens/', api.TokenCreateApi.as_view(), name='auth-token'),
@ -32,6 +35,7 @@ urlpatterns = [
path('password/reset-code/', api.UserResetPasswordSendCodeApi.as_view(), name='reset-password-code'),
path('password/verify/', api.UserPasswordVerifyApi.as_view(), name='user-password-verify'),
path('login-confirm-ticket/status/', api.TicketStatusApi.as_view(), name='login-confirm-ticket-status'),
path('user-session/', api.UserSessionApi.as_view(), name='user-session'),
]
urlpatterns += router.urls + passkey_urlpatterns

View File

@ -49,6 +49,12 @@ urlpatterns = [
path('feishu/qr/bind/callback/', views.FeiShuQRBindCallbackView.as_view(), name='feishu-qr-bind-callback'),
path('feishu/qr/login/callback/', views.FeiShuQRLoginCallbackView.as_view(), name='feishu-qr-login-callback'),
path('lark/bind/start/', views.LarkEnableStartView.as_view(), name='lark-bind-start'),
path('lark/qr/bind/', views.LarkQRBindView.as_view(), name='lark-qr-bind'),
path('lark/qr/login/', views.LarkQRLoginView.as_view(), name='lark-qr-login'),
path('lark/qr/bind/callback/', views.LarkQRBindCallbackView.as_view(), name='lark-qr-bind-callback'),
path('lark/qr/login/callback/', views.LarkQRLoginCallbackView.as_view(), name='lark-qr-login-callback'),
path('slack/bind/start/', views.SlackEnableStartView.as_view(), name='slack-bind-start'),
path('slack/qr/bind/', views.SlackQRBindView.as_view(), name='slack-qr-bind'),
path('slack/qr/login/', views.SlackQRLoginView.as_view(), name='slack-qr-login'),

View File

@ -1,8 +1,9 @@
# -*- coding: utf-8 -*-
#
from .login import *
from .mfa import *
from .wecom import *
from .dingtalk import *
from .feishu import *
from .lark import *
from .login import *
from .mfa import *
from .slack import *
from .wecom import *

View File

@ -1,8 +1,8 @@
from functools import lru_cache
from django.conf import settings
from django.db.utils import IntegrityError
from django.contrib.auth import logout as auth_logout
from django.db.utils import IntegrityError
from django.utils.module_loading import import_string
from django.utils.translation import gettext_lazy as _
from django.views import View
@ -12,8 +12,8 @@ from authentication import errors
from authentication.mixins import AuthMixin
from authentication.notifications import OAuthBindMessage
from common.utils import get_logger
from common.utils.django import reverse, get_object_or_none
from common.utils.common import get_request_ip
from common.utils.django import reverse, get_object_or_none
from users.models import User
from users.signal_handlers import check_only_allow_exist_user_auth
from .mixins import FlashMessageMixin
@ -83,7 +83,15 @@ class BaseLoginCallbackView(AuthMixin, FlashMessageMixin, IMClientMixin, View):
if not self.verify_state():
return self.get_verify_state_failed_response(redirect_url)
user_id, other_info = self.client.get_user_id_by_code(code)
try:
user_id, other_info = self.client.get_user_id_by_code(code)
except Exception:
response = self.get_failed_response(
login_url, title=self.msg_client_err,
msg=self.msg_not_found_user_from_client_err
)
return response
if not user_id:
# 正常流程不会出这个错误hack 行为
err = self.msg_not_found_user_from_client_err

Some files were not shown because too many files have changed in this diff Show More