1.5.7 Merge to dev (#3766)

* [Update] 暂存，优化解决不了问题

* [Update] 待续（小白）

* [Update] 修改asset user

* [Update] 计划再次更改

* [Update] 修改asset user

* [Update] 暂存与喜爱

* [Update] Add id in

* [Update] 阶段性完成ops task该做

* [Update] 修改asset user api

* [Update] 修改asset user 任务，查看认证等

* [Update] 基本完成asset user改造

* [Update] dynamic user only allow 1

* [Update] 修改asset user task

* [Update] 修改node admin user task api

* [Update] remove file header license

* [Update] 添加sftp root

* [Update] 暂存

* [Update] 暂存

* [Update] 修改翻译

* [Update] 修改系统用户改为同名后，用户名改为空

* [Update] 基本完成CAS调研

* [Update] 支持cas server

* [Update] 支持cas server

* [Update] 添加requirements

* [Update] 为方便调试添加mysql ipython到包中

* [Update] 添加huaweiyun翻译

* [Update] 增加下载session 录像

* [Update] 只有第一次通知replay离线的使用方法

* [Update] 暂存一下

* [Bugfix] 获取系统用户信息报错

* [Bugfix] 修改system user info

* [Update] 改成清理10天git status

* [Update] 修改celery日志保留时间

* [Update]修复部分pip包依赖的版本不兼容问题 (#3672)

* [Update] 修复用户更新页面会清空用户public_key的问题

* Fix broken dependencies

Co-authored-by: BaiJiangJie <32935519+BaiJiangJie@users.noreply.github.com>

* [Update] 修改获取系统用户auth info

* [Update] Remove log

* [Bugfix] 修复sftp home设置的bug

* [Update] 授权的系统用户添加sftp root

* [Update] 修改系统用户关联的用户

* [Update] 修改placeholder

* [Update] 优化获取授权的系统用户

* [Update] 修改tasks

* [Update] tree service update

* [Update] 暂存

* [Update] 基本完成用户授权树和资产树改造

* [Update] Dashbaord perf

* [update] Add huawei cloud sdk requirements

* [Updte] 优化dashboard页面

* [Update] system user auth info 添加id

* [Update] 修改系统用户serializer

* [Update] 优化api

* [Update] LDAP Test Util (#3720)

* [Update] LDAPTestUtil 1

* [Update] LDAPTestUtil 2

* [Update] LDAPTestUtil 3

* [Update] LDAPTestUtil 4

* [Update] LDAPTestUtil 5

* [Update] LDAPTestUtil 6

* [Update] LDAPTestUtil 7

* [Update] session 已添加is success，并且添加display serializer

* [Bugfix] 修复无法删除空节点的bug

* [Update] 命令记录分组织显示

* [Update] Session is_success 添加迁移文件

* [Update] 批量命令添加org_id

* [Update] 修复一些文案，修改不绑定MFA，不能ssh登录

* [Update] 修改replay api, 返回session信息

* [Update] 解决无效es导致访问命令记录页面失败的问题

* [Update] 拆分profile view

* [Update] 修改一个翻译

* [Update] 修改aysnc api框架

* [Update] 命令列表添加risk level

* [Update] 完成录像打包下载

* [Update] 更改登陆otp页面

* [Update] 修改command 存储redis_level

* [Update] 修改翻译

* [Update] 修改系统用户的用户列表字段

* [Update] 使用新logo和统一Jumpserver为JumpServer

* [Update] 优化cloud task

* [Update] 统一period task

* [Update] 统一period form serializer字段

* [Update] 修改period task

* [Update] 修改资产网关信息

* [Update] 用户授权资产树资产信息添加domain

* [Update] 修改翻译

* [Update] 测试可连接性

* 1.5.7 bai (#3764)

* [Update] 修复index页面Bug；修复测试资产用户可连接性问题；

* [Update] 修改测试资产用户可连接

* [Bugfix] 修复backends问题

* [Update] 修改marksafe依赖版本

* [Update] 修改测试资产用户可连接性

* [Update] 修改检测服务器性能时获取percent值

* [Update] 更新依赖boto3=1.12.14

Co-authored-by: Yanzhe Lee <lee.yanzhe@yanzhe.org>
Co-authored-by: BaiJiangJie <32935519+BaiJiangJie@users.noreply.github.com>
Co-authored-by: Bai <bugatti_it@163.com>

apps/applications/hands.py

@@ -6,7 +6,7 @@
 
     Other module of this app shouldn't connect with other app.
 
-    :copyright: (c) 2014-2018 by Jumpserver Team.
+    :copyright: (c) 2014-2018 by JumpServer Team.
     :license: GPL v2, see LICENSE for more details.
 """
 

apps/assets/api/asset.py

@@ -23,8 +23,8 @@ from ..filters import AssetByNodeFilterBackend, LabelFilterBackend
 logger = get_logger(__file__)
 __all__ = [
     'AssetViewSet', 'AssetPlatformRetrieveApi',
-    'AssetRefreshHardwareApi', 'AssetAdminUserTestApi',
-    'AssetGatewayApi', 'AssetPlatformViewSet',
+    'AssetGatewayListApi', 'AssetPlatformViewSet',
+    'AssetTaskCreateApi',
 ]
 
 
@@ -36,7 +36,10 @@ class AssetViewSet(OrgBulkModelViewSet):
     filter_fields = ("hostname", "ip", "systemuser__id", "admin_user__id")
     search_fields = ("hostname", "ip")
     ordering_fields = ("hostname", "ip", "port", "cpu_cores")
-    serializer_class = serializers.AssetSerializer
+    serializer_classes = {
+        'default': serializers.AssetSerializer,
+        'display': serializers.AssetDisplaySerializer,
+    }
     permission_classes = (IsOrgAdminOrAppUser,)
     extra_filter_backends = [AssetByNodeFilterBackend, LabelFilterBackend]
 
@@ -80,53 +83,40 @@ class AssetPlatformViewSet(ModelViewSet):
             self.permission_denied(
                 request, message={"detail": "Internal platform"}
             )
-
         return super().check_object_permissions(request, obj)
 
 
-class AssetRefreshHardwareApi(generics.RetrieveAPIView):
-    """
-    Refresh asset hardware info
-    """
+class AssetTaskCreateApi(generics.CreateAPIView):
     model = Asset
-    serializer_class = serializers.AssetSerializer
+    serializer_class = serializers.AssetTaskSerializer
     permission_classes = (IsOrgAdmin,)
 
-    def retrieve(self, request, *args, **kwargs):
-        asset_id = kwargs.get('pk')
-        asset = get_object_or_404(Asset, pk=asset_id)
-        task = update_asset_hardware_info_manual.delay(asset)
-        return Response({"task": task.id})
-
-
-class AssetAdminUserTestApi(generics.RetrieveAPIView):
-    """
-    Test asset admin user assets_connectivity
-    """
-    model = Asset
-    permission_classes = (IsOrgAdmin,)
-    serializer_class = serializers.TaskIDSerializer
+    def get_object(self):
+        pk = self.kwargs.get("pk")
+        instance = get_object_or_404(Asset, pk=pk)
+        return instance
 
-    def retrieve(self, request, *args, **kwargs):
-        asset_id = kwargs.get('pk')
-        asset = get_object_or_404(Asset, pk=asset_id)
-        task = test_asset_connectivity_manual.delay(asset)
-        return Response({"task": task.id})
+    def perform_create(self, serializer):
+        asset = self.get_object()
+        action = serializer.validated_data["action"]
+        if action == "refresh":
+            task = update_asset_hardware_info_manual.delay(asset)
+        else:
+            task = test_asset_connectivity_manual.delay(asset)
+        data = getattr(serializer, '_data', {})
+        data["task"] = task.id
+        setattr(serializer, '_data', data)
 
 
-class AssetGatewayApi(generics.RetrieveAPIView):
+class AssetGatewayListApi(generics.ListAPIView):
     permission_classes = (IsOrgAdminOrAppUser,)
     serializer_class = serializers.GatewayWithAuthSerializer
     model = Asset
 
-    def retrieve(self, request, *args, **kwargs):
-        asset_id = kwargs.get('pk')
+    def get_queryset(self):
+        asset_id = self.kwargs.get('pk')
         asset = get_object_or_404(Asset, pk=asset_id)
-
-        if asset.domain and \
-                asset.domain.gateways.filter(protocol='ssh').exists():
-            gateway = random.choice(asset.domain.gateways.filter(protocol='ssh'))
-            serializer = serializers.GatewayWithAuthSerializer(instance=gateway)
-            return Response(serializer.data)
-        else:
-            return Response({"msg": "Not have gateway"}, status=404)
+        if not asset.domain:
+            return []
+        queryset = asset.domain.gateways.filter(protocol='ssh')
+        return queryset

apps/assets/api/asset_user.py

@@ -1,26 +1,23 @@
 # -*- coding: utf-8 -*-
 #
-
+from django.conf import settings
 from rest_framework.response import Response
-from rest_framework import generics
-from rest_framework import filters
+from rest_framework import generics, filters
 from rest_framework_bulk import BulkModelViewSet
-from django.shortcuts import get_object_or_404
-from django.http import Http404
-from django.conf import settings
 
 from common.permissions import IsOrgAdminOrAppUser, NeedMFAVerify
 from common.utils import get_object_or_none, get_logger
 from common.mixins import CommonApiMixin
 from ..backends import AssetUserManager
-from ..models import Asset, Node, SystemUser, AdminUser
+from ..models import Asset, Node, SystemUser
 from .. import serializers
-from ..tasks import test_asset_users_connectivity_manual
+from ..tasks import (
+    test_asset_users_connectivity_manual, push_system_user_a_asset_manual
+)
 
 
 __all__ = [
-    'AssetUserViewSet', 'AssetUserAuthInfoApi', 'AssetUserTestConnectiveApi',
-    'AssetUserExportViewSet',
+    'AssetUserViewSet', 'AssetUserAuthInfoViewSet', 'AssetUserTaskCreateAPI',
 ]
 
 
@@ -34,10 +31,17 @@ class AssetUserFilterBackend(filters.BaseFilterBackend):
             value = request.GET.get(field)
             if not value:
                 continue
-            if field in ("node_id", "system_user_id", "admin_user_id"):
+            if field == "node_id":
+                value = get_object_or_none(Node, pk=value)
+                kwargs["node"] = value
                 continue
+            elif field == "asset_id":
+                field = "asset"
             kwargs[field] = value
-        return queryset.filter(**kwargs)
+        if kwargs:
+            queryset = queryset.filter(**kwargs)
+        logger.debug("Filter {}".format(kwargs))
+        return queryset
 
 
 class AssetUserSearchBackend(filters.BaseFilterBackend):
@@ -45,71 +49,62 @@ class AssetUserSearchBackend(filters.BaseFilterBackend):
         value = request.GET.get('search')
         if not value:
             return queryset
-        _queryset = AssetUserManager.none()
-        for field in view.search_fields:
-            if field in ("node_id", "system_user_id", "admin_user_id"):
-                continue
-            _queryset |= queryset.filter(**{field: value})
-        return _queryset.distinct()
+        queryset = queryset.search(value)
+        return queryset
+
+
+class AssetUserLatestFilterBackend(filters.BaseFilterBackend):
+    def filter_queryset(self, request, queryset, view):
+        latest = request.GET.get('latest') == '1'
+        if latest:
+            queryset = queryset.distinct()
+        return queryset
 
 
 class AssetUserViewSet(CommonApiMixin, BulkModelViewSet):
-    serializer_class = serializers.AssetUserSerializer
+    serializer_classes = {
+        'default': serializers.AssetUserWriteSerializer,
+        'list': serializers.AssetUserReadSerializer,
+        'retrieve': serializers.AssetUserReadSerializer,
+    }
     permission_classes = [IsOrgAdminOrAppUser]
-    http_method_names = ['get', 'post']
     filter_fields = [
-        "id", "ip", "hostname", "username", "asset_id", "node_id",
-        "system_user_id", "admin_user_id"
+        "id", "ip", "hostname", "username",
+        "asset_id", "node_id",
+        "prefer", "prefer_id",
     ]
-    search_fields = filter_fields
-    filter_backends = (
-        filters.OrderingFilter,
+    search_fields = ["ip", "hostname", "username"]
+    filter_backends = [
         AssetUserFilterBackend, AssetUserSearchBackend,
-    )
+        AssetUserLatestFilterBackend,
+    ]
 
     def allow_bulk_destroy(self, qs, filtered):
         return False
 
-    def get_queryset(self):
-        # 尽可能先返回更少的数据
-        username = self.request.GET.get('username')
-        asset_id = self.request.GET.get('asset_id')
-        node_id = self.request.GET.get('node_id')
-        admin_user_id = self.request.GET.get("admin_user_id")
-        system_user_id = self.request.GET.get("system_user_id")
+    def get_object(self):
+        pk = self.kwargs.get("pk")
+        queryset = self.get_queryset()
+        obj = queryset.get(id=pk)
+        return obj
 
-        kwargs = {}
-        assets = None
+    def get_exception_handler(self):
+        def handler(e, context):
+            return Response({"error": str(e)}, status=400)
+        return handler
+
+    def perform_destroy(self, instance):
+        manager = AssetUserManager()
+        manager.delete(instance)
 
+    def get_queryset(self):
         manager = AssetUserManager()
-        if system_user_id:
-            system_user = get_object_or_404(SystemUser, id=system_user_id)
-            assets = system_user.get_all_assets()
-            username = system_user.username
-        elif admin_user_id:
-            admin_user = get_object_or_404(AdminUser, id=admin_user_id)
-            assets = admin_user.assets.all()
-            username = admin_user.username
-            manager.prefer('admin_user')
-
-        if asset_id:
-            asset = get_object_or_404(Asset, id=asset_id)
-            assets = [asset]
-        elif node_id:
-            node = get_object_or_404(Node, id=node_id)
-            assets = node.get_all_assets()
-
-        if username:
-            kwargs['username'] = username
-        if assets is not None:
-            kwargs['assets'] = assets
-
-        queryset = manager.filter(**kwargs)
+        queryset = manager.all()
         return queryset
 
 
-class AssetUserExportViewSet(AssetUserViewSet):
-    serializer_class = serializers.AssetUserExportSerializer
+class AssetUserAuthInfoViewSet(AssetUserViewSet):
+    serializer_classes = {"default": serializers.AssetUserAuthInfoSerializer}
     http_method_names = ['get']
     permission_classes = [IsOrgAdminOrAppUser]
 
@@ -119,66 +114,31 @@ class AssetUserExportViewSet(AssetUserViewSet):
         return super().get_permissions()
 
 
-class AssetUserAuthInfoApi(generics.RetrieveAPIView):
-    serializer_class = serializers.AssetUserAuthInfoSerializer
-    permission_classes = [IsOrgAdminOrAppUser]
-
-    def get_permissions(self):
-        if settings.SECURITY_VIEW_AUTH_NEED_MFA:
-            self.permission_classes = [IsOrgAdminOrAppUser, NeedMFAVerify]
-        return super().get_permissions()
-
-    def get_object(self):
-        query_params = self.request.query_params
-        username = query_params.get('username')
-        asset_id = query_params.get('asset_id')
-        prefer = query_params.get("prefer")
-        asset = get_object_or_none(Asset, pk=asset_id)
-        try:
-            manger = AssetUserManager()
-            instance = manger.get(username, asset, prefer=prefer)
-        except Exception as e:
-            raise Http404("Not found")
-        else:
-            return instance
-
-
-class AssetUserTestConnectiveApi(generics.RetrieveAPIView):
-    """
-    Test asset users connective
-    """
+class AssetUserTaskCreateAPI(generics.CreateAPIView):
     permission_classes = (IsOrgAdminOrAppUser,)
-    serializer_class = serializers.TaskIDSerializer
+    serializer_class = serializers.AssetUserTaskSerializer
+    filter_backends = AssetUserViewSet.filter_backends
+    filter_fields = AssetUserViewSet.filter_fields
 
     def get_asset_users(self):
-        username = self.request.GET.get('username')
-        asset_id = self.request.GET.get('asset_id')
-        prefer = self.request.GET.get("prefer")
-        asset = get_object_or_none(Asset, pk=asset_id)
         manager = AssetUserManager()
-        asset_users = manager.filter(username=username, assets=[asset], prefer=prefer)
-        return asset_users
+        queryset = manager.all()
+        for cls in self.filter_backends:
+            queryset = cls().filter_queryset(self.request, queryset, self)
+        return list(queryset)
 
-    def retrieve(self, request, *args, **kwargs):
+    def perform_create(self, serializer):
         asset_users = self.get_asset_users()
-        prefer = self.request.GET.get("prefer")
-        kwargs = {}
-        if prefer == "admin_user":
-            kwargs["run_as_admin"] = True
-        task = test_asset_users_connectivity_manual.delay(asset_users, **kwargs)
-        return Response({"task": task.id})
-
-
-class AssetUserPushApi(generics.CreateAPIView):
-    """
-    Test asset users connective
-    """
-    serializer_class = serializers.AssetUserPushSerializer
-    permission_classes = (IsOrgAdminOrAppUser,)
-
-    def create(self, request, *args, **kwargs):
-        serializer = self.get_serializer(data=request.data)
-        serializer.is_valid(raise_exception=True)
-        asset = serializer.validated_data["asset"]
-        username = serializer.validated_data["username"]
-        pass
+        # action = serializer.validated_data["action"]
+        # only this
+        # if action == "test":
+        task = test_asset_users_connectivity_manual.delay(asset_users)
+        data = getattr(serializer, '_data', {})
+        data["task"] = task.id
+        setattr(serializer, '_data', data)
+        return task
+
+    def get_exception_handler(self):
+        def handler(e, context):
+            return Response({"error": str(e)}, status=400)
+        return handler

apps/assets/api/node.py

@@ -1,24 +1,11 @@
 # ~*~ coding: utf-8 ~*~
-# Copyright (C) 2014-2018 Beijing DuiZhan Technology Co.,Ltd. All Rights Reserved.
-#
-# Licensed under the GNU General Public License v2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#    http://www.gnu.org/licenses/gpl-2.0.html
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
 
+from collections import namedtuple
 from rest_framework import status
 from rest_framework.serializers import ValidationError
-from rest_framework.views import APIView
 from rest_framework.response import Response
 from django.utils.translation import ugettext_lazy as _
-from django.shortcuts import get_object_or_404
+from django.shortcuts import get_object_or_404, Http404
 
 from common.utils import get_logger, get_object_or_none
 from common.tree import TreeNodeSerializer
@@ -27,7 +14,8 @@ from orgs.mixins import generics
 from ..hands import IsOrgAdmin
 from ..models import Node
 from ..tasks import (
-    update_assets_hardware_info_util, test_asset_connectivity_util
+    update_node_assets_hardware_info_manual,
+    test_node_assets_connectivity_manual,
 )
 from .. import serializers
 
@@ -36,9 +24,9 @@ logger = get_logger(__file__)
 __all__ = [
     'NodeViewSet', 'NodeChildrenApi', 'NodeAssetsApi',
     'NodeAddAssetsApi', 'NodeRemoveAssetsApi', 'NodeReplaceAssetsApi',
-    'NodeAddChildrenApi', 'RefreshNodeHardwareInfoApi',
-    'TestNodeConnectiveApi', 'NodeListAsTreeApi',
-    'NodeChildrenAsTreeApi', 'RefreshNodesCacheApi',
+    'NodeAddChildrenApi', 'NodeListAsTreeApi',
+    'NodeChildrenAsTreeApi',
+    'NodeTaskCreateApi',
 ]
 
 
@@ -64,9 +52,9 @@ class NodeViewSet(OrgModelViewSet):
 
     def destroy(self, request, *args, **kwargs):
         node = self.get_object()
-        if node.has_children_or_contains_assets():
-            msg = _("Deletion failed and the node contains children or assets")
-            return Response(data={'msg': msg}, status=status.HTTP_403_FORBIDDEN)
+        if node.has_children_or_has_assets():
+            error = _("Deletion failed and the node contains children or assets")
+            return Response(data={'error': error}, status=status.HTTP_403_FORBIDDEN)
         return super().destroy(request, *args, **kwargs)
 
 
@@ -261,41 +249,41 @@ class NodeReplaceAssetsApi(generics.UpdateAPIView):
             asset.nodes.set([instance])
 
 
-class RefreshNodeHardwareInfoApi(APIView):
+class NodeTaskCreateApi(generics.CreateAPIView):
     model = Node
+    serializer_class = serializers.NodeTaskSerializer
     permission_classes = (IsOrgAdmin,)
 
-    def get(self, request, *args, **kwargs):
-        node_id = kwargs.get('pk')
-        node = get_object_or_404(self.model, id=node_id)
-        assets = node.get_all_assets()
-        # task_name = _("更新节点资产硬件信息: {}".format(node.name))
-        task_name = _("Update node asset hardware information: {}").format(node.name)
-        task = update_assets_hardware_info_util.delay(assets, task_name=task_name)
-        return Response({"task": task.id})
-
-
-class TestNodeConnectiveApi(APIView):
-    permission_classes = (IsOrgAdmin,)
-    model = Node
-
-    def get(self, request, *args, **kwargs):
-        node_id = kwargs.get('pk')
-        node = get_object_or_404(self.model, id=node_id)
-        assets = node.get_all_assets()
-        # task_name = _("测试节点下资产是否可连接: {}".format(node.name))
-        task_name = _("Test if the assets under the node are connectable: {}".format(node.name))
-        task = test_asset_connectivity_util.delay(assets, task_name=task_name)
-        return Response({"task": task.id})
-
+    def get_object(self):
+        node_id = self.kwargs.get('pk')
+        node = get_object_or_none(self.model, id=node_id)
+        return node
 
-class RefreshNodesCacheApi(APIView):
-    permission_classes = (IsOrgAdmin,)
+    @staticmethod
+    def set_serializer_data(s, task):
+        data = getattr(s, '_data', {})
+        data["task"] = task.id
+        setattr(s, '_data', data)
 
-    def get(self, request, *args, **kwargs):
+    @staticmethod
+    def refresh_nodes_cache():
         Node.refresh_nodes()
-        return Response("Ok")
+        Task = namedtuple('Task', ['id'])
+        task = Task(id="0")
+        return task
+
+    def perform_create(self, serializer):
+        action = serializer.validated_data["action"]
+        node = self.get_object()
+        if action == "refresh_cache" and node is None:
+            task = self.refresh_nodes_cache()
+            self.set_serializer_data(serializer, task)
+            return
+        if node is None:
+            raise Http404()
+        if action == "refresh":
+            task = update_node_assets_hardware_info_manual.delay(node)
+        else:
+            task = test_node_assets_connectivity_manual.delay(node)
+        self.set_serializer_data(serializer, task)
 
-    def delete(self, *args, **kwargs):
-        self.get(*args, **kwargs)
-        return Response(status=204)

apps/assets/api/system_user.py

@@ -1,42 +1,25 @@
 # ~*~ coding: utf-8 ~*~
-# Copyright (C) 2014-2018 Beijing DuiZhan Technology Co.,Ltd. All Rights Reserved.
-#
-# Licensed under the GNU General Public License v2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#    http://www.gnu.org/licenses/gpl-2.0.html
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
 from django.shortcuts import get_object_or_404
 from rest_framework.response import Response
-from django.db.models import Count
 
-from common.serializers import CeleryTaskSerializer
 from common.utils import get_logger
 from common.permissions import IsOrgAdmin, IsOrgAdminOrAppUser, IsAppUser
 from orgs.mixins.api import OrgBulkModelViewSet
 from orgs.mixins import generics
+from orgs.utils import tmp_to_org
 from ..models import SystemUser, Asset
 from .. import serializers
+from ..serializers import SystemUserWithAuthInfoSerializer
 from ..tasks import (
     push_system_user_to_assets_manual, test_system_user_connectivity_manual,
-    push_system_user_a_asset_manual, test_system_user_connectivity_a_asset,
+    push_system_user_a_asset_manual,
 )
 
 
 logger = get_logger(__file__)
 __all__ = [
     'SystemUserViewSet', 'SystemUserAuthInfoApi', 'SystemUserAssetAuthInfoApi',
-    'SystemUserPushApi', 'SystemUserTestConnectiveApi',
-    'SystemUserAssetsListView', 'SystemUserPushToAssetApi',
-    'SystemUserTestAssetConnectivityApi', 'SystemUserCommandFilterRuleListApi',
-
+    'SystemUserCommandFilterRuleListApi', 'SystemUserTaskApi',
 ]
 
 
@@ -48,13 +31,12 @@ class SystemUserViewSet(OrgBulkModelViewSet):
     filter_fields = ("name", "username")
     search_fields = filter_fields
     serializer_class = serializers.SystemUserSerializer
+    serializer_classes = {
+        'default': serializers.SystemUserSerializer,
+        'list': serializers.SystemUserListSerializer,
+    }
     permission_classes = (IsOrgAdminOrAppUser,)
 
-    def get_queryset(self):
-        queryset = super().get_queryset()
-        queryset = queryset.annotate(_assets_amount=Count('assets'))
-        return queryset
-
 
 class SystemUserAuthInfoApi(generics.RetrieveUpdateDestroyAPIView):
     """
@@ -62,7 +44,7 @@ class SystemUserAuthInfoApi(generics.RetrieveUpdateDestroyAPIView):
     """
     model = SystemUser
     permission_classes = (IsOrgAdminOrAppUser,)
-    serializer_class = serializers.SystemUserAuthSerializer
+    serializer_class = SystemUserWithAuthInfoSerializer
 
     def destroy(self, request, *args, **kwargs):
         instance = self.get_object()
@@ -75,88 +57,61 @@ class SystemUserAssetAuthInfoApi(generics.RetrieveAPIView):
     Get system user with asset auth info
     """
     model = SystemUser
-    permission_classes = (IsAppUser,)
-    serializer_class = serializers.SystemUserAuthSerializer
+    permission_classes = (IsOrgAdminOrAppUser,)
+    serializer_class = SystemUserWithAuthInfoSerializer
+
+    def get_exception_handler(self):
+        def handler(e, context):
+            return Response({"error": str(e)}, status=400)
+        return handler
 
     def get_object(self):
         instance = super().get_object()
-        aid = self.kwargs.get('aid')
-        asset = get_object_or_404(Asset, pk=aid)
-        instance.load_specific_asset_auth(asset)
-        return instance
-
-
-class SystemUserPushApi(generics.RetrieveAPIView):
-    """
-    Push system user to cluster assets api
-    """
-    model = SystemUser
-    permission_classes = (IsOrgAdmin,)
-    serializer_class = CeleryTaskSerializer
+        username = instance.username
+        if instance.username_same_with_user:
+            username = self.request.query_params.get("username")
+        asset_id = self.kwargs.get('aid')
+        asset = get_object_or_404(Asset, pk=asset_id)
 
-    def retrieve(self, request, *args, **kwargs):
-        system_user = self.get_object()
-        nodes = system_user.nodes.all()
-        for node in nodes:
-            system_user.assets.add(*tuple(node.get_all_assets()))
-        task = push_system_user_to_assets_manual.delay(system_user)
-        return Response({"task": task.id})
+        with tmp_to_org(asset.org_id):
+            instance.load_asset_special_auth(asset=asset, username=username)
+            return instance
 
 
-class SystemUserTestConnectiveApi(generics.RetrieveAPIView):
-    """
-    Push system user to cluster assets api
-    """
-    model = SystemUser
+class SystemUserTaskApi(generics.CreateAPIView):
     permission_classes = (IsOrgAdmin,)
-    serializer_class = CeleryTaskSerializer
-
-    def retrieve(self, request, *args, **kwargs):
-        system_user = self.get_object()
+    serializer_class = serializers.SystemUserTaskSerializer
+
+    def do_push(self, system_user, asset=None):
+        if asset is None:
+            task = push_system_user_to_assets_manual.delay(system_user)
+        else:
+            username = self.request.query_params.get('username')
+            task = push_system_user_a_asset_manual.delay(
+                system_user, asset, username=username
+            )
+        return task
+
+    @staticmethod
+    def do_test(system_user, asset=None):
         task = test_system_user_connectivity_manual.delay(system_user)
-        return Response({"task": task.id})
-
-
-class SystemUserAssetsListView(generics.ListAPIView):
-    permission_classes = (IsOrgAdmin,)
-    serializer_class = serializers.AssetSimpleSerializer
-    filter_fields = ("hostname", "ip")
-    http_method_names = ['get']
-    search_fields = filter_fields
+        return task
 
     def get_object(self):
         pk = self.kwargs.get('pk')
         return get_object_or_404(SystemUser, pk=pk)
 
-    def get_queryset(self):
+    def perform_create(self, serializer):
+        action = serializer.validated_data["action"]
+        asset = serializer.validated_data.get('asset')
         system_user = self.get_object()
-        return system_user.assets.all()
-
-
-class SystemUserPushToAssetApi(generics.RetrieveAPIView):
-    model = SystemUser
-    permission_classes = (IsOrgAdmin,)
-    serializer_class = serializers.TaskIDSerializer
-
-    def retrieve(self, request, *args, **kwargs):
-        system_user = self.get_object()
-        asset_id = self.kwargs.get('aid')
-        asset = get_object_or_404(Asset, id=asset_id)
-        task = push_system_user_a_asset_manual.delay(system_user, asset)
-        return Response({"task": task.id})
-
-
-class SystemUserTestAssetConnectivityApi(generics.RetrieveAPIView):
-    model = SystemUser
-    permission_classes = (IsOrgAdmin,)
-    serializer_class = serializers.TaskIDSerializer
-
-    def retrieve(self, request, *args, **kwargs):
-        system_user = self.get_object()
-        asset_id = self.kwargs.get('aid')
-        asset = get_object_or_404(Asset, id=asset_id)
-        task = test_system_user_connectivity_a_asset.delay(system_user, asset)
-        return Response({"task": task.id})
+        if action == 'push':
+            task = self.do_push(system_user, asset)
+        else:
+            task = self.do_test(system_user, asset)
+        data = getattr(serializer, '_data', {})
+        data["task"] = task.id
+        setattr(serializer, '_data', data)
 
 
 class SystemUserCommandFilterRuleListApi(generics.ListAPIView):

apps/assets/api/system_user_relation.py

@@ -8,10 +8,13 @@ from orgs.mixins.api import OrgBulkModelViewSet
 from orgs.utils import current_org
 from .. import models, serializers
 
-__all__ = ['SystemUserAssetRelationViewSet', 'SystemUserNodeRelationViewSet']
+__all__ = [
+    'SystemUserAssetRelationViewSet', 'SystemUserNodeRelationViewSet',
+    'SystemUserUserRelationViewSet',
+]
 
 
-class RelationMixin(OrgBulkModelViewSet):
+class RelationMixin:
     def get_queryset(self):
         queryset = self.model.objects.all()
         org_id = current_org.org_id()
@@ -24,7 +27,11 @@ class RelationMixin(OrgBulkModelViewSet):
         return queryset
 
 
-class SystemUserAssetRelationViewSet(RelationMixin):
+class BaseRelationViewSet(RelationMixin, OrgBulkModelViewSet):
+    pass
+
+
+class SystemUserAssetRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserAssetRelationSerializer
     model = models.SystemUser.assets.through
     permission_classes = (IsOrgAdmin,)
@@ -47,7 +54,7 @@ class SystemUserAssetRelationViewSet(RelationMixin):
         return queryset
 
 
-class SystemUserNodeRelationViewSet(RelationMixin):
+class SystemUserNodeRelationViewSet(BaseRelationViewSet):
     serializer_class = serializers.SystemUserNodeRelationSerializer
     model = models.SystemUser.nodes.through
     permission_classes = (IsOrgAdmin,)
@@ -63,3 +70,27 @@ class SystemUserNodeRelationViewSet(RelationMixin):
         queryset = queryset \
             .annotate(node_key=F('node__key'))
         return queryset
+
+
+class SystemUserUserRelationViewSet(BaseRelationViewSet):
+    serializer_class = serializers.SystemUserUserRelationSerializer
+    model = models.SystemUser.users.through
+    permission_classes = (IsOrgAdmin,)
+    filterset_fields = [
+        'id', 'user', 'systemuser',
+    ]
+    search_fields = [
+        "user__username", "user__name",
+        "systemuser__name", "systemuser__username",
+    ]
+
+    def get_queryset(self):
+        queryset = super().get_queryset()
+        queryset = queryset.annotate(
+            user_display=Concat(
+                F('user__name'), Value('('),
+                F('user__username'), Value(')')
+            )
+        )
+        return queryset
+

apps/assets/backends/admin_user.py

@@ -1,10 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-
-from ..models import AdminUser
-from .asset_user import AssetUserBackend
-
-
-class AdminUserBackend(AssetUserBackend):
-    model = AdminUser
-    backend = 'AdminUser'

apps/assets/backends/asset_user.py

@@ -1,58 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-from collections import defaultdict
-from .base import BaseBackend
-
-
-class AssetUserBackend(BaseBackend):
-    model = None
-    backend = "AssetUser"
-
-    @classmethod
-    def filter_queryset_more(cls, queryset):
-        return queryset
-
-    @classmethod
-    def filter(cls, username=None, assets=None, **kwargs):
-        queryset = cls.model.objects.all()
-        prefer_id = kwargs.get('prefer_id')
-        if prefer_id:
-            queryset = queryset.filter(id=prefer_id)
-            instances = cls.construct_authbook_objects(queryset, assets)
-            return instances
-        if username:
-            queryset = queryset.filter(username=username)
-        if assets:
-            queryset = queryset.filter(assets__in=assets).distinct()
-
-        queryset = cls.filter_queryset_more(queryset)
-        instances = cls.construct_authbook_objects(queryset, assets)
-        return instances
-
-    @classmethod
-    def construct_authbook_objects(cls, asset_users, assets):
-        instances = []
-        assets_user_assets_map = defaultdict(set)
-        if isinstance(asset_users, list):
-            assets_user_assets_map = {
-                asset_user.id: asset_user.assets.values_list('id', flat=True)
-                for asset_user in asset_users
-            }
-        else:
-            assets_user_assets = asset_users.values_list('id', 'assets')
-            for i, asset_id in assets_user_assets:
-                assets_user_assets_map[i].add(asset_id)
-
-        for asset_user in asset_users:
-            if not assets:
-                related_assets = asset_user.assets.all()
-            else:
-                assets_map = {a.id: a for a in assets}
-                related_assets = [
-                    assets_map.get(i) for i in assets_user_assets_map.get(asset_user.id) if i in assets_map
-                ]
-            for asset in related_assets:
-                instance = asset_user.construct_to_authbook(asset)
-                instance.backend = cls.backend
-                instances.append(instance)
-        return instances

apps/assets/backends/base.py

@@ -1,94 +1,48 @@
 # -*- coding: utf-8 -*-
 #
-import uuid
 from abc import abstractmethod
 
+from ..models import Asset
+
 
 class BaseBackend:
-    @classmethod
     @abstractmethod
-    def filter(cls, username=None, assets=None, latest=True, prefer=None, prefer_id=None):
-        """
-        :param username: 用户名
-        :param assets: <Asset>对象
-        :param latest: 是否是最新记录
-        :param prefer: 优先使用
-        :param prefer_id: 使用id
-        :return: 元素为<AuthBook>的可迭代对象(<list> or <QuerySet>)
-        """
+    def all(self):
         pass
 
+    @abstractmethod
+    def filter(self, username=None, hostname=None, ip=None, assets=None,
+               node=None, prefer_id=None, **kwargs):
+        pass
 
-class AssetUserQuerySet(list):
-    def order_by(self, *ordering):
-        _ordering = []
-        reverse = False
-        for i in ordering:
-            if i[0] == '-':
-                reverse = True
-                i = i[1:]
-            _ordering.append(i)
-        self.sort(key=lambda obj: [getattr(obj, j) for j in _ordering], reverse=reverse)
-        return self
-
-    def filter_in(self, kwargs):
-        in_kwargs = {}
-        queryset = []
-        for k, v in kwargs.items():
-            if len(v) == 0:
-                return self
-            if k.find("__in") >= 0:
-                _k = k.split('__')[0]
-                in_kwargs[_k] = v
-            else:
-                in_kwargs[k] = v
-        for k in in_kwargs:
-            kwargs.pop(k, None)
-
-        if len(in_kwargs) == 0:
-            return self
-        for i in self:
-            matched = False
-            for k, v in in_kwargs.items():
-                attr = getattr(i, k, None)
-                # 如果属性或者value中是uuid,则转换成string
-                if isinstance(v[0], uuid.UUID):
-                    v = [str(i) for i in v]
-                if isinstance(attr, uuid.UUID):
-                    attr = str(attr)
-                if attr in v:
-                    matched = True
-            if matched:
-                queryset.append(i)
-        return AssetUserQuerySet(queryset)
-
-    def filter_equal(self, kwargs):
-        def filter_it(obj):
-            wanted = []
-            real = []
-            for k, v in kwargs.items():
-                wanted.append(v)
-                value = getattr(obj, k, None)
-                if isinstance(value, uuid.UUID):
-                    value = str(value)
-                real.append(value)
-            return wanted == real
-        kwargs = {k: v for k, v in kwargs.items() if k.find('__in') == -1}
-        if len(kwargs) > 0:
-            queryset = AssetUserQuerySet([i for i in self if filter_it(i)])
-        else:
-            queryset = self
-        return queryset
+    @abstractmethod
+    def search(self, item):
+        pass
 
-    def filter(self, **kwargs):
-        queryset = self.filter_in(kwargs).filter_equal(kwargs)
-        return queryset
+    @abstractmethod
+    def get_queryset(self):
+        pass
 
-    def distinct(self):
-        items = list(set(self))
-        self[:] = items
-        return self
+    @abstractmethod
+    def delete(self, union_id):
+        pass
 
-    def __or__(self, other):
-        self.extend(other)
-        return self
+    @staticmethod
+    def qs_to_values(qs):
+        values = qs.values(
+            'hostname', 'ip', "asset_id",
+            'username', 'password', 'private_key', 'public_key',
+            'score', 'version',
+            "asset_username", "union_id",
+            'date_created', 'date_updated',
+            'org_id', 'backend',
+        )
+        return values
+
+    @staticmethod
+    def make_assets_as_id(assets):
+        if not assets:
+            return []
+        if isinstance(assets[0], Asset):
+            assets = [a.id for a in assets]
+        return assets

apps/assets/backends/db.py

@@ -1,29 +1,317 @@
 # -*- coding: utf-8 -*-
 #
+from functools import reduce
+from django.db.models import F, CharField, Value, IntegerField, Q, Count
+from django.db.models.functions import Concat
 
-from ..models import AuthBook
+from common.utils import get_object_or_none
+from orgs.utils import current_org
+from ..models import AuthBook, SystemUser, Asset, AdminUser
 from .base import BaseBackend
 
 
-class AuthBookBackend(BaseBackend):
-    @classmethod
-    def filter(cls, username=None, assets=None, latest=True, **kwargs):
-        queryset = AuthBook.objects.all()
-        if username is not None:
-            queryset = queryset.filter(username=username)
-        if assets:
-            queryset = queryset.filter(asset__in=assets)
-        if latest:
-            queryset = queryset.latest_version()
-        return queryset
-
-    @classmethod
-    def create(cls, **kwargs):
-        auth_info = {
-            'password': kwargs.pop('password', ''),
-            'public_key': kwargs.pop('public_key', ''),
-            'private_key': kwargs.pop('private_key', '')
-        }
-        obj = AuthBook.objects.create(**kwargs)
-        obj.set_auth(**auth_info)
-        return obj
+class DBBackend(BaseBackend):
+    union_id_length = 2
+
+    def __init__(self, queryset=None):
+        if queryset is None:
+            queryset = self.all()
+        self.queryset = queryset
+
+    def _clone(self):
+        return self.__class__(self.queryset)
+
+    def all(self):
+        return AuthBook.objects.none()
+
+    def count(self):
+        return self.queryset.count()
+
+    def get_queryset(self):
+        return self.queryset
+
+    def delete(self, union_id):
+        cleaned_union_id = union_id.split('_')
+        # 如果union_id通不过本检查，代表可能不是本backend, 应该返回空
+        if not self._check_union_id(union_id, cleaned_union_id):
+            return
+        return self._perform_delete_by_union_id(cleaned_union_id)
+
+    def _perform_delete_by_union_id(self, union_id_cleaned):
+        pass
+
+    def filter(self, assets=None, node=None, prefer=None, prefer_id=None,
+               union_id=None, id__in=None, **kwargs):
+        clone = self._clone()
+        clone._filter_union_id(union_id)
+        clone._filter_prefer(prefer, prefer_id)
+        clone._filter_node(node)
+        clone._filter_assets(assets)
+        clone._filter_other(kwargs)
+        clone._filter_id_in(id__in)
+        return clone
+
+    def _filter_union_id(self, union_id):
+        if not union_id:
+            return
+        cleaned_union_id = union_id.split('_')
+        # 如果union_id通不过本检查，代表可能不是本backend, 应该返回空
+        if not self._check_union_id(union_id, cleaned_union_id):
+            self.queryset = self.queryset.none()
+            return
+        return self._perform_filter_union_id(union_id, cleaned_union_id)
+
+    def _check_union_id(self, union_id, cleaned_union_id):
+        return union_id and len(cleaned_union_id) == self.union_id_length
+
+    def _perform_filter_union_id(self, union_id, union_id_cleaned):
+        self.queryset = self.queryset.filter(union_id=union_id)
+
+    def _filter_assets(self, assets):
+        assets_id = self.make_assets_as_id(assets)
+        if assets_id:
+            self.queryset = self.queryset.filter(asset_id__in=assets_id)
+
+    def _filter_node(self, node):
+        pass
+
+    def _filter_id_in(self, ids):
+        if ids and isinstance(ids, list):
+            self.queryset = self.queryset.filter(union_id__in=ids)
+
+    @staticmethod
+    def clean_kwargs(kwargs):
+        return {k: v for k, v in kwargs.items() if v}
+
+    def _filter_other(self, kwargs):
+        kwargs = self.clean_kwargs(kwargs)
+        if kwargs:
+            self.queryset = self.queryset.filter(**kwargs)
+
+    def _filter_prefer(self, prefer, prefer_id):
+        pass
+
+    def search(self, item):
+        qs = []
+        for i in ['hostname', 'ip', 'username']:
+            kwargs = {i + '__startswith': item}
+            qs.append(Q(**kwargs))
+        q = reduce(lambda x, y: x | y, qs)
+        clone = self._clone()
+        clone.queryset = clone.queryset.filter(q).distinct()
+        return clone
+
+
+class SystemUserBackend(DBBackend):
+    model = SystemUser.assets.through
+    backend = 'system_user'
+    prefer = backend
+    base_score = 0
+    union_id_length = 2
+
+    def _filter_prefer(self, prefer, prefer_id):
+        if prefer and prefer != self.prefer:
+            self.queryset = self.queryset.none()
+
+        if prefer_id:
+            self.queryset = self.queryset.filter(systemuser__id=prefer_id)
+
+    def _perform_filter_union_id(self, union_id, union_id_cleaned):
+        system_user_id, asset_id = union_id_cleaned
+        self.queryset = self.queryset.filter(
+            asset_id=asset_id, systemuser__id=system_user_id,
+        )
+
+    def _perform_delete_by_union_id(self, union_id_cleaned):
+        system_user_id, asset_id = union_id_cleaned
+        system_user = get_object_or_none(SystemUser, pk=system_user_id)
+        asset = get_object_or_none(Asset, pk=asset_id)
+        if all((system_user, asset)):
+            system_user.assets.remove(asset)
+
+    def _filter_node(self, node):
+        if node:
+            self.queryset = self.queryset.filter(asset__nodes__id=node.id)
+
+    def get_annotate(self):
+        kwargs = dict(
+            hostname=F("asset__hostname"),
+            ip=F("asset__ip"),
+            username=F("systemuser__username"),
+            password=F("systemuser__password"),
+            private_key=F("systemuser__private_key"),
+            public_key=F("systemuser__public_key"),
+            score=F("systemuser__priority") + self.base_score,
+            version=Value(0, IntegerField()),
+            date_created=F("systemuser__date_created"),
+            date_updated=F("systemuser__date_updated"),
+            asset_username=Concat(F("asset__id"), Value("_"),
+                                  F("systemuser__username"),
+                                  output_field=CharField()),
+            union_id=Concat(F("systemuser_id"), Value("_"), F("asset_id"),
+                            output_field=CharField()),
+            org_id=F("asset__org_id"),
+            backend=Value(self.backend, CharField())
+        )
+        return kwargs
+
+    def get_filter(self):
+        return dict(
+            systemuser__username_same_with_user=False,
+        )
+
+    def all(self):
+        kwargs = self.get_annotate()
+        filters = self.get_filter()
+        qs = self.model.objects.all().annotate(**kwargs)
+        if current_org.org_id() is not None:
+            filters['org_id'] = current_org.org_id()
+        qs = qs.filter(**filters)
+        qs = self.qs_to_values(qs)
+        return qs
+
+
+class DynamicSystemUserBackend(SystemUserBackend):
+    backend = 'system_user_dynamic'
+    prefer = 'system_user'
+    union_id_length = 3
+
+    def get_annotate(self):
+        kwargs = super().get_annotate()
+        kwargs.update(dict(
+            username=F("systemuser__users__username"),
+            asset_username=Concat(
+                F("asset__id"), Value("_"),
+                F("systemuser__users__username"),
+                output_field=CharField()
+            ),
+            union_id=Concat(
+                F("systemuser_id"), Value("_"), F("asset_id"),
+                Value("_"), F("systemuser__users__id"),
+                output_field=CharField()
+            ),
+            users_count=Count('systemuser__users'),
+        ))
+        return kwargs
+
+    def _perform_filter_union_id(self, union_id, union_id_cleaned):
+        system_user_id, asset_id, user_id = union_id_cleaned
+        self.queryset = self.queryset.filter(
+            asset_id=asset_id, systemuser_id=system_user_id,
+            union_id=union_id,
+        )
+
+    def _perform_delete_by_union_id(self, union_id_cleaned):
+        system_user_id, asset_id, user_id = union_id_cleaned
+        system_user = get_object_or_none(SystemUser, pk=system_user_id)
+        if not system_user:
+            return
+        system_user.users.remove(user_id)
+        if system_user.users.count() == 0:
+            system_user.assets.remove(asset_id)
+
+    def get_filter(self):
+        return dict(
+            users_count__gt=0,
+            systemuser__username_same_with_user=True
+        )
+
+
+class AdminUserBackend(DBBackend):
+    model = Asset
+    backend = 'admin_user'
+    prefer = backend
+    base_score = 200
+
+    def _filter_prefer(self, prefer, prefer_id):
+        if prefer and prefer != self.backend:
+            self.queryset = self.queryset.none()
+        if prefer_id:
+            self.queryset = self.queryset.filter(admin_user__id=prefer_id)
+
+    def _filter_node(self, node):
+        if node:
+            self.queryset = self.queryset.filter(nodes__id=node.id)
+
+    def _perform_filter_union_id(self, union_id, union_id_cleaned):
+        admin_user_id, asset_id = union_id_cleaned
+        self.queryset = self.queryset.filter(
+            id=asset_id, admin_user_id=admin_user_id,
+        )
+
+    def _perform_delete_by_union_id(self, union_id_cleaned):
+        raise PermissionError("Could remove asset admin user")
+
+    def all(self):
+        qs = self.model.objects.all().annotate(
+            asset_id=F("id"),
+            username=F("admin_user__username"),
+            password=F("admin_user__password"),
+            private_key=F("admin_user__private_key"),
+            public_key=F("admin_user__public_key"),
+            score=Value(self.base_score, IntegerField()),
+            version=Value(0, IntegerField()),
+            date_updated=F("admin_user__date_updated"),
+            asset_username=Concat(F("id"), Value("_"), F("admin_user__username"), output_field=CharField()),
+            union_id=Concat(F("admin_user_id"), Value("_"), F("id"), output_field=CharField()),
+            backend=Value(self.backend, CharField()),
+        )
+        qs = self.qs_to_values(qs)
+        return qs
+
+
+class AuthbookBackend(DBBackend):
+    model = AuthBook
+    backend = 'db'
+    prefer = backend
+    base_score = 400
+
+    def _filter_node(self, node):
+        if node:
+            self.queryset = self.queryset.filter(asset__nodes__id=node.id)
+
+    def _filter_prefer(self, prefer, prefer_id):
+        if not prefer or not prefer_id:
+            return
+        if prefer.lower() == "admin_user":
+            model = AdminUser
+        elif prefer.lower() == "system_user":
+            model = SystemUser
+        else:
+            self.queryset = self.queryset.none()
+            return
+        obj = get_object_or_none(model, pk=prefer_id)
+        if obj is None:
+            self.queryset = self.queryset.none()
+            return
+        username = obj.get_username()
+        if isinstance(username, str):
+            self.queryset = self.queryset.filter(username=username)
+        # dynamic system user return more username
+        else:
+            self.queryset = self.queryset.filter(username__in=username)
+
+    def _perform_filter_union_id(self, union_id, union_id_cleaned):
+        authbook_id, asset_id = union_id_cleaned
+        self.queryset = self.queryset.filter(
+            id=authbook_id, asset_id=asset_id,
+        )
+
+    def _perform_delete_by_union_id(self, union_id_cleaned):
+        authbook_id, asset_id = union_id_cleaned
+        authbook = get_object_or_none(AuthBook, pk=authbook_id)
+        if authbook.is_latest:
+            raise PermissionError("Latest version could be delete")
+        AuthBook.objects.filter(id=authbook_id).delete()
+
+    def all(self):
+        qs = self.model.objects.all().annotate(
+            hostname=F("asset__hostname"),
+            ip=F("asset__ip"),
+            score=F('version') + self.base_score,
+            asset_username=Concat(F("asset__id"), Value("_"), F("username"), output_field=CharField()),
+            union_id=Concat(F("id"), Value("_"), F("asset_id"), output_field=CharField()),
+            backend=Value(self.backend, CharField()),
+        )
+        qs = self.qs_to_values(qs)
+        return qs

apps/assets/backends/manager.py

@@ -1,110 +1,162 @@
 # -*- coding: utf-8 -*-
 #
+from itertools import chain, groupby
 from django.core.exceptions import MultipleObjectsReturned, ObjectDoesNotExist
 
-from .base import AssetUserQuerySet
-from .db import AuthBookBackend
-from .system_user import SystemUserBackend
-from .admin_user import AdminUserBackend
+from orgs.utils import current_org
+from common.utils import get_logger, lazyproperty
+from common.struct import QuerySetChain
+
+from ..models import AssetUser, AuthBook
+from .db import (
+    AuthbookBackend, SystemUserBackend, AdminUserBackend,
+    DynamicSystemUserBackend
+)
+
+logger = get_logger(__name__)
 
 
 class NotSupportError(Exception):
     pass
 
 
-class AssetUserManager:
-    """
-    资产用户管理器
-    """
+class AssetUserQueryset:
     ObjectDoesNotExist = ObjectDoesNotExist
     MultipleObjectsReturned = MultipleObjectsReturned
-    NotSupportError = NotSupportError
-    MSG_NOT_EXIST = '{} Object matching query does not exist'
-    MSG_MULTIPLE = '{} get() returned more than one object ' \
-                   '-- it returned {}!'
 
-    backends = (
-        ('db', AuthBookBackend),
-        ('system_user', SystemUserBackend),
-        ('admin_user', AdminUserBackend),
-    )
+    def __init__(self, backends=()):
+        self.backends = backends
+        self._distinct_queryset = None
+
+    def backends_queryset(self):
+        return [b.get_queryset() for b in self.backends]
+
+    @lazyproperty
+    def backends_counts(self):
+        return [b.count() for b in self.backends]
+
+    def filter(self, hostname=None, ip=None, username=None,
+               assets=None, asset=None, node=None,
+               id=None, prefer_id=None, prefer=None, id__in=None):
+        if not assets and asset:
+            assets = [asset]
+
+        kwargs = dict(
+            hostname=hostname, ip=ip, username=username,
+            assets=assets, node=node, prefer=prefer, prefer_id=prefer_id,
+            id__in=id__in, union_id=id,
+        )
+        logger.debug("Filter: {}".format(kwargs))
+        backends = []
+        for backend in self.backends:
+            clone = backend.filter(**kwargs)
+            backends.append(clone)
+        return self._clone(backends)
+
+    def _clone(self, backends=None):
+        if backends is None:
+            backends = self.backends
+        return self.__class__(backends)
+
+    def search(self, item):
+        backends = []
+        for backend in self.backends:
+            new = backend.search(item)
+            backends.append(new)
+        return self._clone(backends)
+
+    def distinct(self):
+        logger.debug("Distinct asset user queryset")
+        queryset_chain = chain(*(backend.get_queryset() for backend in self.backends))
+        queryset_sorted = sorted(
+            queryset_chain,
+            key=lambda item: (item["asset_username"], item["score"]),
+            reverse=True,
+        )
+        results = groupby(queryset_sorted, key=lambda item: item["asset_username"])
+        final = [next(result[1]) for result in results]
+        self._distinct_queryset = final
+        return self
 
-    _prefer = "system_user"
-
-    def filter(self, username=None, assets=None, latest=True, prefer=None, prefer_id=None):
-        if assets is not None and not assets:
-            return AssetUserQuerySet([])
-
-        if prefer:
-            self._prefer = prefer
-
-        instances_map = {}
-        instances = []
-        for name, backend in self.backends:
-            # if name != "db":
-            #     continue
-            _instances = backend.filter(
-                username=username, assets=assets, latest=latest,
-                prefer=self._prefer, prefer_id=prefer_id,
-            )
-            instances_map[name] = _instances
-
-        # 如果不是获取最新版本，就不再merge
-        if not latest:
-            for _instances in instances_map.values():
-                instances.extend(_instances)
-            return AssetUserQuerySet(instances)
-
-        # merge的顺序
-        ordering = ["db"]
-        if self._prefer == "system_user":
-            ordering.extend(["system_user", "admin_user"])
+    def get(self, latest=False, **kwargs):
+        queryset = self.filter(**kwargs)
+        if latest:
+            queryset = queryset.distinct()
+        queryset = list(queryset)
+        count = len(queryset)
+        if count == 1:
+            data = queryset[0]
+            return data
+        elif count > 1:
+            msg = 'Should return 1 record, but get {}'.format(count)
+            raise MultipleObjectsReturned(msg)
         else:
-            ordering.extend(["admin_user", "system_user"])
-        # 根据prefer决定优先使用系统用户或管理用户谁的
-        ordering_instances = [instances_map.get(i, []) for i in ordering]
-        instances = self._merge_instances(*ordering_instances)
-        return AssetUserQuerySet(instances)
-
-    def get(self, username, asset, **kwargs):
-        instances = self.filter(username, assets=[asset], **kwargs)
-        if len(instances) == 1:
-            return instances[0]
-        elif len(instances) == 0:
-            self.raise_does_not_exist(self.__class__.__name__)
-        else:
-            self.raise_multiple_return(self.__class__.__name__, len(instances))
-
-    def raise_does_not_exist(self, name):
-        raise self.ObjectDoesNotExist(self.MSG_NOT_EXIST.format(name))
+            msg = 'No record found(org is {})'.format(current_org.name)
+            raise ObjectDoesNotExist(msg)
 
-    def raise_multiple_return(self, name, length):
-        raise self.MultipleObjectsReturned(self.MSG_MULTIPLE.format(name, length))
+    def get_latest(self, **kwargs):
+        return self.get(latest=True, **kwargs)
 
     @staticmethod
-    def create(**kwargs):
-        instance = AuthBookBackend.create(**kwargs)
-        return instance
+    def to_asset_user(data):
+        obj = AssetUser()
+        for k, v in data.items():
+            setattr(obj, k, v)
+        return obj
+
+    @property
+    def queryset(self):
+        if self._distinct_queryset is not None:
+            return self._distinct_queryset
+        return QuerySetChain(self.backends_queryset())
+
+    def count(self):
+        if self._distinct_queryset is not None:
+            return len(self._distinct_queryset)
+        else:
+            return sum(self.backends_counts)
 
-    def all(self):
-        return self.filter()
+    def __getitem__(self, ndx):
+        return self.queryset.__getitem__(ndx)
 
-    def prefer(self, s):
-        self._prefer = s
+    def __iter__(self):
+        self._data = iter(self.queryset)
         return self
 
-    @staticmethod
-    def none():
-        return AssetUserQuerySet()
+    def __next__(self):
+        return self.to_asset_user(next(self._data))
+
+
+class AssetUserManager:
+    support_backends = (
+        ('db', AuthbookBackend),
+        ('system_user', SystemUserBackend),
+        ('admin_user', AdminUserBackend),
+        ('system_user_dynamic', DynamicSystemUserBackend),
+    )
+
+    def __init__(self):
+        self.backends = [backend() for name, backend in self.support_backends]
+        self._queryset = AssetUserQueryset(self.backends)
+
+    def all(self):
+        return self._queryset
+
+    def delete(self, obj):
+        name_backends_map = dict(self.support_backends)
+        backend_name = obj.backend
+        backend_cls = name_backends_map.get(backend_name)
+        union_id = obj.union_id
+        if backend_cls:
+            backend_cls().delete(union_id)
+        else:
+            raise ObjectDoesNotExist("Not backend found")
 
     @staticmethod
-    def _merge_instances(*args):
-        instances = list(args[0])
-        keywords = [obj.keyword for obj in instances]
-
-        for _instances in args[1:]:
-            need_merge_instances = [obj for obj in _instances if obj.keyword not in keywords]
-            need_merge_keywords = [obj.keyword for obj in need_merge_instances]
-            instances.extend(need_merge_instances)
-            keywords.extend(need_merge_keywords)
-        return instances
+    def create(**kwargs):
+        authbook = AuthBook(**kwargs)
+        authbook.save()
+        return authbook
+
+    def __getattr__(self, item):
+        return getattr(self._queryset, item)

apps/assets/backends/system_user.py

@@ -1,30 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-
-import itertools
-
-from assets.models import SystemUser
-from .asset_user import AssetUserBackend
-
-
-class SystemUserBackend(AssetUserBackend):
-    model = SystemUser
-    backend = 'SystemUser'
-
-    @classmethod
-    def filter_queryset_more(cls, queryset):
-        queryset = cls._distinct_system_users_by_username(queryset)
-        return queryset
-
-    @classmethod
-    def _distinct_system_users_by_username(cls, system_users):
-        system_users = sorted(
-            system_users,
-            key=lambda su: (su.username, su.priority, su.date_updated),
-            reverse=True,
-        )
-        results = itertools.groupby(system_users, key=lambda su: su.username)
-        system_users = [next(result[1]) for result in results]
-        return system_users
-
-

apps/assets/backends/utils.py

@@ -3,14 +3,5 @@
 
 # from django.conf import settings
 
-from .db import AuthBookBackend
 # from .vault import VaultBackend
 
-
-def get_backend():
-    default_backend = AuthBookBackend
-
-    # if settings.BACKEND_ASSET_USER_AUTH_VAULT:
-    #     return VaultBackend
-
-    return default_backend

apps/assets/backends/vault.py

@@ -1,11 +1,4 @@
 # -*- coding: utf-8 -*-
 #
 
-from .base import BaseBackend
 
-
-class VaultBackend(BaseBackend):
-
-    @classmethod
-    def filter(cls, username=None, asset=None, latest=True):
-        pass

apps/assets/forms/asset.py

@@ -1,12 +1,13 @@
 # -*- coding: utf-8 -*-
 #
+from itertools import groupby
 from django import forms
 from django.utils.translation import gettext_lazy as _
 
 from common.utils import get_logger
 from orgs.mixins.forms import OrgModelForm
 
-from ..models import Asset
+from ..models import Asset, Platform
 
 
 logger = get_logger(__file__)
@@ -42,9 +43,26 @@ class AssetCreateUpdateForm(OrgModelForm):
             ]
         nodes_field.choices = nodes_choices
 
+    @staticmethod
+    def sorted_platform(platform):
+        if platform['base'] == 'Other':
+            return 'zz'
+        return platform['base']
+
     def set_platform_to_name(self):
+        choices = []
+        platforms = Platform.objects.all().values('name', 'base')
+        platforms_sorted = sorted(platforms, key=self.sorted_platform)
+        platforms_grouped = groupby(platforms_sorted, key=lambda x: x['base'])
+        for i in platforms_grouped:
+            base = i[0]
+            grouped = sorted(i[1], key=lambda x: x['name'])
+            grouped = [(j['name'], j['name']) for j in grouped]
+            choices.append(
+                (base, grouped)
+            )
         platform_field = self.fields['platform']
-        platform_field.to_field_name = 'name'
+        platform_field.choices = choices
         if self.instance:
             self.initial['platform'] = self.instance.platform.name
 

apps/assets/forms/user.py

@@ -88,7 +88,9 @@ class SystemUserForm(OrgModelForm, PasswordAndKeyAuthForm):
         fields = [
             'name', 'username', 'protocol', 'auto_generate_key',
             'password', 'private_key', 'auto_push', 'sudo',
+            'username_same_with_user',
             'comment', 'shell', 'priority', 'login_mode', 'cmd_filters',
+            'sftp_root',
         ]
         widgets = {
             'name': forms.TextInput(attrs={'placeholder': _('Name')}),
@@ -97,11 +99,17 @@ class SystemUserForm(OrgModelForm, PasswordAndKeyAuthForm):
                 'class': 'select2', 'data-placeholder': _('Command filter')
             }),
         }
+        labels = {
+            'username_same_with_user': _("Username same with user"),
+        }
         help_texts = {
             'auto_push': _('Auto push system user to asset'),
             'priority': _('1-100, High level will be using login asset as default, '
                           'if user was granted more than 2 system user'),
             'login_mode': _('If you choose manual login mode, you do not '
                             'need to fill in the username and password.'),
-            'sudo': _("Use comma split multi command, ex: /bin/whoami,/bin/ifconfig")
+            'sudo': _("Use comma split multi command, ex: /bin/whoami,/bin/ifconfig"),
+            'sftp_root': _("SFTP root dir, tmp, home or custom"),
+            'username_same_with_user': _("Username is dynamic, When connect asset, using current user's username"),
+            # 'username_same_with_user': _("用户名是动态的，登录资产时使用当前用户的用户名登录"),
         }

apps/assets/hands.py

@@ -6,7 +6,7 @@
 
     Other module of this app shouldn't connect with other app.
 
-    :copyright: (c) 2014-2018 by Jumpserver Team.
+    :copyright: (c) 2014-2018 by JumpServer Team.
     :license: GPL v2, see LICENSE for more details.
 """
 

apps/assets/migrations/0047_assetuser.py

@@ -0,0 +1,24 @@
+# Generated by Django 2.2.7 on 2020-01-06 07:34
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0046_auto_20191218_1705'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='AssetUser',
+            fields=[
+            ],
+            options={
+                'proxy': True,
+                'indexes': [],
+                'constraints': [],
+            },
+            bases=('assets.authbook',),
+        ),
+    ]

apps/assets/migrations/0048_auto_20191230_1512.py

@@ -0,0 +1,35 @@
+# Generated by Django 2.2.7 on 2019-12-30 07:12
+
+from django.conf import settings
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+        ('assets', '0047_assetuser'),
+    ]
+
+    operations = [
+        migrations.RemoveField(
+            model_name='authbook',
+            name='is_active',
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='username_same_with_user',
+            field=models.BooleanField(default=False, verbose_name='Username same with user'),
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='users',
+            field=models.ManyToManyField(blank=True, to=settings.AUTH_USER_MODEL, verbose_name='Users'),
+        ),
+        migrations.AddField(
+            model_name='systemuser',
+            name='groups',
+            field=models.ManyToManyField(blank=True, to='users.UserGroup',
+                                         verbose_name='User groups'),
+        ),
+    ]

apps/assets/migrations/0049_systemuser_sftp_root.py

@@ -0,0 +1,18 @@
+# Generated by Django 2.2.7 on 2020-01-19 07:29
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('assets', '0048_auto_20191230_1512'),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name='systemuser',
+            name='sftp_root',
+            field=models.CharField(default='tmp', max_length=128, verbose_name='SFTP Root'),
+        ),
+    ]

apps/assets/models/__init__.py

@@ -1,6 +1,8 @@
+from .base import *
 from .asset import *
 from .label import Label
 from .user import *
+from .asset_user import *
 from .cluster import *
 from .group import *
 from .domain import *

apps/assets/models/asset.py

@@ -14,6 +14,7 @@ from django.utils.translation import ugettext_lazy as _
 from common.fields.model import JsonDictTextField
 from common.utils import lazyproperty
 from orgs.mixins.models import OrgModelMixin, OrgManager
+from .base import ConnectivityMixin
 from .utils import Connectivity
 
 __all__ = ['Asset', 'ProtocolsMixin', 'Platform']
@@ -40,10 +41,11 @@ def default_node():
 
 
 class AssetManager(OrgManager):
-    def get_queryset(self):
-        return super().get_queryset().annotate(
-            platform_base=models.F('platform__base')
-        )
+    # def get_queryset(self):
+    #     return super().get_queryset().annotate(
+    #         platform_base=models.F('platform__base')
+    #     )
+    pass
 
 
 class AssetQuerySet(models.QuerySet):
@@ -243,6 +245,13 @@ class Asset(ProtocolsMixin, NodesRelationMixin, OrgModelMixin):
     def platform_base(self):
         return self.platform.base
 
+    @lazyproperty
+    def admin_user_username(self):
+        """求可连接性时，直接用用户名去取，避免再查一次admin user
+        serializer 中直接通过annotate方式返回了这个
+        """
+        return self.admin_user.username
+
     def is_windows(self):
         return self.platform.is_windows()
 
@@ -275,9 +284,11 @@ class Asset(ProtocolsMixin, NodesRelationMixin, OrgModelMixin):
     def connectivity(self):
         if self._connectivity:
             return self._connectivity
-        if not self.admin_user:
+        if not self.admin_user_username:
             return Connectivity.unknown()
-        connectivity = self.admin_user.get_asset_connectivity(self)
+        connectivity = ConnectivityMixin.get_asset_username_connectivity(
+            self, self.admin_user_username
+        )
         return connectivity
 
     @connectivity.setter
@@ -290,7 +301,7 @@ class Asset(ProtocolsMixin, NodesRelationMixin, OrgModelMixin):
         if not self.admin_user:
             return {}
 
-        self.admin_user.load_specific_asset_auth(self)
+        self.admin_user.load_asset_special_auth(self)
         info = {
             'username': self.admin_user.username,
             'password': self.admin_user.password,

apps/assets/models/asset_user.py

@@ -0,0 +1,14 @@
+# -*- coding: utf-8 -*-
+#
+from .authbook import AuthBook
+
+
+class AssetUser(AuthBook):
+    hostname = ""
+    ip = ""
+    backend = ""
+    union_id = ""
+    asset_username = ""
+
+    class Meta:
+        proxy = True

apps/assets/models/authbook.py

@@ -5,26 +5,24 @@ from django.db import models
 from django.utils.translation import ugettext_lazy as _
 
 from orgs.mixins.models import OrgManager
-from .base import AssetUser
+from .base import BaseUser
 
 __all__ = ['AuthBook']
 
 
 class AuthBookQuerySet(models.QuerySet):
-
     def latest_version(self):
-        return self.filter(is_latest=True).filter(is_active=True)
+        return self.filter(is_latest=True)
 
 
 class AuthBookManager(OrgManager):
     pass
 
 
-class AuthBook(AssetUser):
+class AuthBook(BaseUser):
     asset = models.ForeignKey('assets.Asset', on_delete=models.CASCADE, verbose_name=_('Asset'))
     is_latest = models.BooleanField(default=False, verbose_name=_('Latest version'))
     version = models.IntegerField(default=1, verbose_name=_('Version'))
-    is_active = models.BooleanField(default=True, verbose_name=_("Is active"))
 
     objects = AuthBookManager.from_queryset(AuthBookQuerySet)()
     backend = "db"

apps/assets/models/base.py

@@ -12,98 +12,29 @@ from django.utils.translation import ugettext_lazy as _
 from django.conf import settings
 
 from common.utils import (
-    signer, ssh_key_string_to_obj, ssh_key_gen, get_logger
+    ssh_key_string_to_obj, ssh_key_gen, get_logger, lazyproperty
 )
 from common.validators import alphanumeric
 from common import fields
 from orgs.mixins.models import OrgModelMixin
-from .utils import private_key_validator, Connectivity
+from .utils import Connectivity
 
 
 logger = get_logger(__file__)
 
 
-class AssetUser(OrgModelMixin):
-    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
-    name = models.CharField(max_length=128, verbose_name=_('Name'))
-    username = models.CharField(max_length=32, blank=True, verbose_name=_('Username'), validators=[alphanumeric], db_index=True)
-    password = fields.EncryptCharField(max_length=256, blank=True, null=True, verbose_name=_('Password'))
-    private_key = fields.EncryptTextField(blank=True, null=True, verbose_name=_('SSH private key'))
-    public_key = fields.EncryptTextField(blank=True, null=True, verbose_name=_('SSH public key'))
-    comment = models.TextField(blank=True, verbose_name=_('Comment'))
-    date_created = models.DateTimeField(auto_now_add=True, verbose_name=_("Date created"))
-    date_updated = models.DateTimeField(auto_now=True, verbose_name=_("Date updated"))
-    created_by = models.CharField(max_length=128, null=True, verbose_name=_('Created by'))
-
+class ConnectivityMixin:
     CONNECTIVITY_ASSET_CACHE_KEY = "ASSET_USER_{}_{}_ASSET_CONNECTIVITY"
     CONNECTIVITY_AMOUNT_CACHE_KEY = "ASSET_USER_{}_{}_CONNECTIVITY_AMOUNT"
-    ASSETS_AMOUNT_CACHE_KEY = "ASSET_USER_{}_ASSETS_AMOUNT"
     ASSET_USER_CACHE_TIME = 3600 * 24
-
-    _prefer = "system_user"
-    _assets_amount = None
-
-    @property
-    def private_key_obj(self):
-        if self.private_key:
-            return ssh_key_string_to_obj(self.private_key, password=self.password)
-        else:
-            return None
-
-    @property
-    def private_key_file(self):
-        if not self.private_key_obj:
-            return None
-        project_dir = settings.PROJECT_DIR
-        tmp_dir = os.path.join(project_dir, 'tmp')
-        key_name = '.' + md5(self.private_key.encode('utf-8')).hexdigest()
-        key_path = os.path.join(tmp_dir, key_name)
-        if not os.path.exists(key_path):
-            self.private_key_obj.write_private_key_file(key_path)
-            os.chmod(key_path, 0o400)
-        return key_path
-
-    @property
-    def public_key_obj(self):
-        if self.public_key:
-            try:
-                return sshpubkeys.SSHKey(self.public_key)
-            except TabError:
-                pass
-        return None
+    id = ''
+    username = ''
 
     @property
     def part_id(self):
         i = '-'.join(str(self.id).split('-')[:3])
         return i
 
-    def get_private_key(self):
-        if not self.private_key_obj:
-            return None
-        string_io = io.StringIO()
-        self.private_key_obj.write_private_key(string_io)
-        private_key = string_io.getvalue()
-        return private_key
-
-    def get_related_assets(self):
-        assets = self.assets.all()
-        return assets
-
-    def set_auth(self, password=None, private_key=None, public_key=None):
-        update_fields = []
-        if password:
-            self.password = password
-            update_fields.append('password')
-        if private_key:
-            self.private_key = private_key
-            update_fields.append('private_key')
-        if public_key:
-            self.public_key = public_key
-            update_fields.append('public_key')
-
-        if update_fields:
-            self.save(update_fields=update_fields)
-
     def set_connectivity(self, summary):
         unreachable = summary.get('dark', {}).keys()
         reachable = summary.get('contacted', {}).keys()
@@ -150,20 +81,10 @@ class AssetUser(OrgModelMixin):
             cache.set(cache_key, amount, self.ASSET_USER_CACHE_TIME)
         return amount
 
-    @property
-    def assets_amount(self):
-        if self._assets_amount is not None:
-            return self._assets_amount
-        cache_key = self.ASSETS_AMOUNT_CACHE_KEY.format(self.id)
-        cached = cache.get(cache_key)
-        if not cached:
-            cached = self.get_related_assets().count()
-            cache.set(cache_key, cached, self.ASSET_USER_CACHE_TIME)
-        return cached
-
-    def expire_assets_amount(self):
-        cache_key = self.ASSETS_AMOUNT_CACHE_KEY.format(self.id)
-        cache.delete(cache_key)
+    @classmethod
+    def get_asset_username_connectivity(cls, asset, username):
+        key = cls.CONNECTIVITY_ASSET_CACHE_KEY.format(username, asset.id)
+        return Connectivity.get(key)
 
     def get_asset_connectivity(self, asset):
         key = self.get_asset_connectivity_key(asset)
@@ -176,28 +97,103 @@ class AssetUser(OrgModelMixin):
         key = self.get_asset_connectivity_key(asset)
         Connectivity.set(key, c)
 
-    def get_asset_user(self, asset):
+
+class AuthMixin:
+    private_key = ''
+    password = ''
+    public_key = ''
+    username = ''
+    _prefer = 'system_user'
+
+    @property
+    def private_key_obj(self):
+        if self.private_key:
+            key_obj = ssh_key_string_to_obj(self.private_key, password=self.password)
+            return key_obj
+        else:
+            return None
+
+    @property
+    def private_key_file(self):
+        if not self.private_key_obj:
+            return None
+        project_dir = settings.PROJECT_DIR
+        tmp_dir = os.path.join(project_dir, 'tmp')
+        key_name = '.' + md5(self.private_key.encode('utf-8')).hexdigest()
+        key_path = os.path.join(tmp_dir, key_name)
+        if not os.path.exists(key_path):
+            self.private_key_obj.write_private_key_file(key_path)
+            os.chmod(key_path, 0o400)
+        return key_path
+
+    def get_private_key(self):
+        if not self.private_key_obj:
+            return None
+        string_io = io.StringIO()
+        self.private_key_obj.write_private_key(string_io)
+        private_key = string_io.getvalue()
+        return private_key
+
+    @property
+    def public_key_obj(self):
+        if self.public_key:
+            try:
+                return sshpubkeys.SSHKey(self.public_key)
+            except TabError:
+                pass
+        return None
+
+    def set_auth(self, password=None, private_key=None, public_key=None):
+        update_fields = []
+        if password:
+            self.password = password
+            update_fields.append('password')
+        if private_key:
+            self.private_key = private_key
+            update_fields.append('private_key')
+        if public_key:
+            self.public_key = public_key
+            update_fields.append('public_key')
+
+        if update_fields:
+            self.save(update_fields=update_fields)
+
+    def has_special_auth(self, asset=None):
+        from .authbook import AuthBook
+        queryset = AuthBook.objects.filter(username=self.username)
+        if asset:
+            queryset = queryset.filter(asset=asset)
+        return queryset.exists()
+
+    def get_asset_user(self, asset, username=None):
         from ..backends import AssetUserManager
+        if username is None:
+            username = self.username
         try:
-            manager = AssetUserManager().prefer(self._prefer)
-            other = manager.get(username=self.username, asset=asset, prefer_id=self.id)
+            manager = AssetUserManager()
+            other = manager.get_latest(
+                username=username, asset=asset,
+                prefer_id=self.id, prefer=self._prefer,
+            )
             return other
         except Exception as e:
             logger.error(e, exc_info=True)
             return None
 
-    def load_specific_asset_auth(self, asset):
-        instance = self.get_asset_user(asset)
+    def load_asset_special_auth(self, asset=None, username=None):
+        if not asset:
+            return self
+
+        instance = self.get_asset_user(asset, username=username)
         if instance:
             self._merge_auth(instance)
 
     def _merge_auth(self, other):
         if other.password:
             self.password = other.password
-        if other.public_key:
-            self.public_key = other.public_key
-        if other.private_key:
+        if other.public_key or other.private_key:
             self.private_key = other.private_key
+            self.public_key = other.public_key
 
     def clear_auth(self):
         self.password = ''
@@ -216,19 +212,57 @@ class AssetUser(OrgModelMixin):
         )
         return private_key, public_key
 
-    def auto_gen_auth(self):
-        password = str(uuid.uuid4())
-        private_key, public_key = ssh_key_gen(
-            username=self.username
-        )
+    def auto_gen_auth(self, password=True, key=True):
+        _password = None
+        _private_key = None
+        _public_key = None
+
+        if password:
+            _password = self.gen_password()
+        if key:
+            _private_key, _public_key = self.gen_key(self.username)
         self.set_auth(
-            password=password, private_key=private_key,
-            public_key=public_key
+            password=_password, private_key=_private_key,
+            public_key=_public_key
         )
 
-    def auto_gen_auth_password(self):
-        password = str(uuid.uuid4())
-        self.set_auth(password=password)
+
+class BaseUser(OrgModelMixin, AuthMixin, ConnectivityMixin):
+    id = models.UUIDField(default=uuid.uuid4, primary_key=True)
+    name = models.CharField(max_length=128, verbose_name=_('Name'))
+    username = models.CharField(max_length=32, blank=True, verbose_name=_('Username'), validators=[alphanumeric], db_index=True)
+    password = fields.EncryptCharField(max_length=256, blank=True, null=True, verbose_name=_('Password'))
+    private_key = fields.EncryptTextField(blank=True, null=True, verbose_name=_('SSH private key'))
+    public_key = fields.EncryptTextField(blank=True, null=True, verbose_name=_('SSH public key'))
+    comment = models.TextField(blank=True, verbose_name=_('Comment'))
+    date_created = models.DateTimeField(auto_now_add=True, verbose_name=_("Date created"))
+    date_updated = models.DateTimeField(auto_now=True, verbose_name=_("Date updated"))
+    created_by = models.CharField(max_length=128, null=True, verbose_name=_('Created by'))
+
+    ASSETS_AMOUNT_CACHE_KEY = "ASSET_USER_{}_ASSETS_AMOUNT"
+    ASSET_USER_CACHE_TIME = 600
+
+    _prefer = "system_user"
+
+    def get_related_assets(self):
+        assets = self.assets.filter(org_id=self.org_id)
+        return assets
+
+    def get_username(self):
+        return self.username
+
+    @lazyproperty
+    def assets_amount(self):
+        cache_key = self.ASSETS_AMOUNT_CACHE_KEY.format(self.id)
+        cached = cache.get(cache_key)
+        if not cached:
+            cached = self.get_related_assets().count()
+            cache.set(cache_key, cached, self.ASSET_USER_CACHE_TIME)
+        return cached
+
+    def expire_assets_amount(self):
+        cache_key = self.ASSETS_AMOUNT_CACHE_KEY.format(self.id)
+        cache.delete(cache_key)
 
     def _to_secret_json(self):
         """Push system user use it"""
@@ -240,26 +274,6 @@ class AssetUser(OrgModelMixin):
             'private_key': self.private_key_file,
         }
 
-    def generate_id_with_asset(self, asset):
-        user_id = [self.part_id]
-        asset_id = str(asset.id).split('-')[3:]
-        ids = user_id + asset_id
-        return '-'.join(ids)
-
-    def construct_to_authbook(self, asset):
-        from . import AuthBook
-        fields = [
-            'name', 'username', 'comment', 'org_id',
-            'password', 'private_key', 'public_key',
-            'date_created', 'date_updated', 'created_by'
-        ]
-        i = self.generate_id_with_asset(asset)
-        obj = AuthBook(id=i, asset=asset, version=0, is_latest=True)
-        for field in fields:
-            value = getattr(self, field)
-            setattr(obj, field, value)
-        return obj
-
     class Meta:
         abstract = True
 

apps/assets/models/domain.py

@@ -10,7 +10,7 @@ from django.db import models
 from django.utils.translation import ugettext_lazy as _
 
 from orgs.mixins.models import OrgModelMixin
-from .base import AssetUser
+from .base import BaseUser
 
 __all__ = ['Domain', 'Gateway']
 
@@ -39,7 +39,7 @@ class Domain(OrgModelMixin):
         return random.choice(self.gateways)
 
 
-class Gateway(AssetUser):
+class Gateway(BaseUser):
     PROTOCOL_SSH = 'ssh'
     PROTOCOL_RDP = 'rdp'
     PROTOCOL_CHOICES = (

apps/assets/models/node.py

@@ -11,9 +11,9 @@ from django.utils.translation import ugettext_lazy as _
 from django.utils.translation import ugettext
 from django.core.cache import cache
 
-from common.utils import get_logger, timeit, lazyproperty
+from common.utils import get_logger, lazyproperty
 from orgs.mixins.models import OrgModelMixin, OrgManager
-from orgs.utils import set_current_org, get_current_org, tmp_to_org
+from orgs.utils import get_current_org, tmp_to_org, current_org
 from orgs.models import Organization
 
 
@@ -26,63 +26,108 @@ class NodeQuerySet(models.QuerySet):
         raise PermissionError("Bulk delete node deny")
 
 
-class TreeMixin:
-    tree_created_time = None
-    tree_updated_time_cache_key = 'NODE_TREE_UPDATED_AT'
-    tree_cache_time = 3600
-    tree_assets_cache_key = 'NODE_TREE_ASSETS_UPDATED_AT'
-    tree_assets_created_time = None
-    _tree_service = None
+class TreeCache:
+    updated_time_cache_key = 'NODE_TREE_UPDATED_AT_{}'
+    cache_time = 3600
+    assets_updated_time_cache_key = 'NODE_TREE_ASSETS_UPDATED_AT_{}'
 
-    @classmethod
-    def tree(cls):
-        from ..utils import TreeService
-        tree_updated_time = cache.get(cls.tree_updated_time_cache_key, 0)
+    def __init__(self, tree, org_id):
         now = time.time()
-        # 什么时候重新初始化 _tree_service
-        if not cls.tree_created_time or \
-                tree_updated_time > cls.tree_created_time:
-            logger.debug("Create node tree")
-            tree = TreeService.new()
-            cls.tree_created_time = now
-            cls.tree_assets_created_time = now
-            cls._tree_service = tree
-            return tree
-        # 是否要重新初始化节点资产
-        node_assets_updated_time = cache.get(cls.tree_assets_cache_key, 0)
-        if not cls.tree_assets_created_time or \
-                node_assets_updated_time > cls.tree_assets_created_time:
-            cls._tree_service.init_assets()
-            cls.tree_assets_created_time = now
-            logger.debug("Refresh node tree assets")
-        return cls._tree_service
+        self.created_time = now
+        self.assets_created_time = now
+        self.tree = tree
+        self.org_id = org_id
+
+    def _has_changed(self, tp="tree"):
+        if tp == "assets":
+            key = self.assets_updated_time_cache_key.format(self.org_id)
+        else:
+            key = self.updated_time_cache_key.format(self.org_id)
+        updated_time = cache.get(key, 0)
+        if updated_time > self.created_time:
+            return True
+        else:
+            return False
 
     @classmethod
-    def refresh_tree(cls, t=None):
-        logger.debug("Refresh node tree")
-        key = cls.tree_updated_time_cache_key
-        ttl = cls.tree_cache_time
+    def set_changed(cls, tp="tree", t=None, org_id=None):
+        if org_id is None:
+            org_id = current_org.id
+        if tp == "assets":
+            key = cls.assets_updated_time_cache_key.format(org_id)
+        else:
+            key = cls.updated_time_cache_key.format(org_id)
+        ttl = cls.cache_time
         if not t:
             t = time.time()
         cache.set(key, t, ttl)
 
+    def tree_has_changed(self):
+        return self._has_changed("tree")
+
+    def set_tree_changed(self, t=None):
+        logger.debug("Set tree tree changed")
+        self.__class__.set_changed(t=t, tp="tree")
+
+    def assets_has_changed(self):
+        return self._has_changed("assets")
+
+    def set_tree_assets_changed(self, t=None):
+        logger.debug("Set tree assets changed")
+        self.__class__.set_changed(t=t, tp="assets")
+
+    def get(self):
+        if self.tree_has_changed():
+            self.renew()
+            return self.tree
+        if self.assets_has_changed():
+            self.tree.init_assets()
+        return self.tree
+
+    def renew(self):
+        new_obj = self.__class__.new(self.org_id)
+        self.tree = new_obj.tree
+        self.created_time = new_obj.created_time
+        self.assets_created_time = new_obj.assets_created_time
+
     @classmethod
-    def refresh_node_assets(cls, t=None):
-        logger.debug("Refresh node assets")
-        key = cls.tree_assets_cache_key
-        ttl = cls.tree_cache_time
-        if not t:
-            t = time.time()
-        cache.set(key, t, ttl)
+    def new(cls, org_id=None):
+        from ..utils import TreeService
+        logger.debug("Create node tree")
+        if not org_id:
+            org_id = current_org.id
+        with tmp_to_org(org_id):
+            tree = TreeService.new()
+            obj = cls(tree, org_id)
+            obj.tree = tree
+            return obj
 
-    @staticmethod
-    def refresh_user_tree_cache():
-        """
-        当节点-节点关系，节点-资产关系发生变化时，应该刷新用户授权树缓存
-        :return:
-        """
-        from perms.utils.asset_permission import AssetPermissionUtilV2
-        AssetPermissionUtilV2.expire_all_user_tree_cache()
+
+class TreeMixin:
+    _org_tree_map = {}
+
+    @classmethod
+    def tree(cls):
+        org_id = current_org.org_id()
+        t = cls.get_local_tree_cache(org_id)
+
+        if t is None:
+            t = TreeCache.new()
+            cls._org_tree_map[org_id] = t
+        return t.get()
+
+    @classmethod
+    def get_local_tree_cache(cls, org_id=None):
+        t = cls._org_tree_map.get(org_id)
+        return t
+
+    @classmethod
+    def refresh_tree(cls, t=None):
+        TreeCache.set_changed(tp="tree", t=t, org_id=current_org.id)
+
+    @classmethod
+    def refresh_node_assets(cls, t=None):
+        TreeCache.set_changed(tp="assets", t=t, org_id=current_org.id)
 
 
 class FamilyMixin:
@@ -376,15 +421,6 @@ class SomeNodesMixin:
             )
             return obj
 
-    @classmethod
-    def empty_node(cls):
-        with tmp_to_org(Organization.system()):
-            defaults = {'value': cls.empty_value}
-            obj, created = cls.objects.get_or_create(
-                defaults=defaults, key=cls.empty_key
-            )
-            return obj
-
     @classmethod
     def default_node(cls):
         with tmp_to_org(Organization.default()):
@@ -413,7 +449,6 @@ class SomeNodesMixin:
     @classmethod
     def initial_some_nodes(cls):
         cls.default_node()
-        cls.empty_node()
         cls.ungrouped_node()
         cls.favorite_node()
 
@@ -523,13 +558,13 @@ class Node(OrgModelMixin, SomeNodesMixin, TreeMixin, FamilyMixin, FullValueMixin
         tree_node = TreeNode(**data)
         return tree_node
 
-    def has_children_or_contains_assets(self):
-        if self.children or self.get_assets():
+    def has_children_or_has_assets(self):
+        if self.children or self.get_assets().exists():
             return True
         return False
 
     def delete(self, using=None, keep_parents=False):
-        if self.has_children_or_contains_assets():
+        if self.has_children_or_has_assets():
             return
         return super().delete(using=using, keep_parents=keep_parents)
 

apps/assets/models/user.py

@@ -4,14 +4,12 @@
 
 import logging
 
-from functools import reduce
 from django.db import models
-from django.db.models import Q
 from django.utils.translation import ugettext_lazy as _
 from django.core.validators import MinValueValidator, MaxValueValidator
 
 from common.utils import signer
-from .base import AssetUser
+from .base import BaseUser
 from .asset import Asset
 
 
@@ -19,7 +17,7 @@ __all__ = ['AdminUser', 'SystemUser']
 logger = logging.getLogger(__name__)
 
 
-class AdminUser(AssetUser):
+class AdminUser(BaseUser):
     """
     A privileged user that ansible can use it to push system user and so on
     """
@@ -87,7 +85,7 @@ class AdminUser(AssetUser):
                 continue
 
 
-class SystemUser(AssetUser):
+class SystemUser(BaseUser):
     PROTOCOL_SSH = 'ssh'
     PROTOCOL_RDP = 'rdp'
     PROTOCOL_TELNET = 'telnet'
@@ -107,9 +105,11 @@ class SystemUser(AssetUser):
         (LOGIN_AUTO, _('Automatic login')),
         (LOGIN_MANUAL, _('Manually login'))
     )
-
+    username_same_with_user = models.BooleanField(default=False, verbose_name=_("Username same with user"))
     nodes = models.ManyToManyField('assets.Node', blank=True, verbose_name=_("Nodes"))
     assets = models.ManyToManyField('assets.Asset', blank=True, verbose_name=_("Assets"))
+    users = models.ManyToManyField('users.User', blank=True, verbose_name=_("Users"))
+    groups = models.ManyToManyField('users.UserGroup', blank=True, verbose_name=_("User groups"))
     priority = models.IntegerField(default=20, verbose_name=_("Priority"), validators=[MinValueValidator(1), MaxValueValidator(100)])
     protocol = models.CharField(max_length=16, choices=PROTOCOL_CHOICES, default='ssh', verbose_name=_('Protocol'))
     auto_push = models.BooleanField(default=True, verbose_name=_('Auto push'))
@@ -117,9 +117,20 @@ class SystemUser(AssetUser):
     shell = models.CharField(max_length=64,  default='/bin/bash', verbose_name=_('Shell'))
     login_mode = models.CharField(choices=LOGIN_MODE_CHOICES, default=LOGIN_AUTO, max_length=10, verbose_name=_('Login mode'))
     cmd_filters = models.ManyToManyField('CommandFilter', related_name='system_users', verbose_name=_("Command filter"), blank=True)
+    sftp_root = models.CharField(default='tmp', max_length=128, verbose_name=_("SFTP Root"))
+    _prefer = 'system_user'
 
     def __str__(self):
-        return '{0.name}({0.username})'.format(self)
+        username = self.username
+        if self.username_same_with_user:
+            username = 'dynamic'
+        return '{0.name}({1})'.format(self, username)
+
+    def get_username(self):
+        if self.username_same_with_user:
+            return list(self.users.values_list('username', flat=True))
+        else:
+            return self.username
 
     @property
     def nodes_amount(self):

apps/assets/serializers/admin_user.py

@@ -55,3 +55,11 @@ class ReplaceNodeAdminUserSerializer(serializers.ModelSerializer):
 
 class TaskIDSerializer(serializers.Serializer):
     task = serializers.CharField(read_only=True)
+
+
+class AssetUserTaskSerializer(serializers.Serializer):
+    ACTION_CHOICES = (
+        ('test', 'test'),
+    )
+    action = serializers.ChoiceField(choices=ACTION_CHOICES, write_only=True)
+    task = serializers.CharField(read_only=True)

apps/assets/serializers/asset.py

@@ -1,8 +1,8 @@
 # -*- coding: utf-8 -*-
 #
-import re
 from rest_framework import serializers
-from django.db.models import Prefetch
+from django.db.models import Prefetch, F
+
 from django.utils.translation import ugettext_lazy as _
 
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
@@ -12,8 +12,9 @@ from .base import ConnectivitySerializer
 
 __all__ = [
     'AssetSerializer', 'AssetSimpleSerializer',
+    'AssetDisplaySerializer',
     'ProtocolsField', 'PlatformSerializer',
-    'AssetDetailSerializer',
+    'AssetDetailSerializer', 'AssetTaskSerializer',
 ]
 
 
@@ -66,8 +67,6 @@ class AssetSerializer(BulkOrgResourceModelSerializer):
         slug_field='name', queryset=Platform.objects.all(), label=_("Platform")
     )
     protocols = ProtocolsField(label=_('Protocols'), required=False)
-    connectivity = ConnectivitySerializer(read_only=True, label=_("Connectivity"))
-
     """
     资产的数据结构
     """
@@ -81,7 +80,7 @@ class AssetSerializer(BulkOrgResourceModelSerializer):
             'cpu_model', 'cpu_count', 'cpu_cores', 'cpu_vcpus', 'memory',
             'disk_total', 'disk_info', 'os', 'os_version', 'os_arch',
             'hostname_raw', 'comment', 'created_by', 'date_created',
-            'hardware_info', 'connectivity',
+            'hardware_info',
         ]
         read_only_fields = (
             'vendor', 'model', 'sn', 'cpu_model', 'cpu_count',
@@ -102,7 +101,8 @@ class AssetSerializer(BulkOrgResourceModelSerializer):
         queryset = queryset.prefetch_related(
             Prefetch('nodes', queryset=Node.objects.all().only('id')),
             Prefetch('labels', queryset=Label.objects.all().only('id')),
-        ).select_related('admin_user', 'domain', 'platform')
+        ).select_related('admin_user', 'domain', 'platform') \
+         .annotate(platform_base=F('platform__base'))
         return queryset
 
     def compatible_with_old_protocol(self, validated_data):
@@ -130,6 +130,28 @@ class AssetSerializer(BulkOrgResourceModelSerializer):
         return super().update(instance, validated_data)
 
 
+class AssetDisplaySerializer(AssetSerializer):
+    connectivity = ConnectivitySerializer(read_only=True, label=_("Connectivity"))
+
+    class Meta(AssetSerializer.Meta):
+        fields = [
+            'id', 'ip', 'hostname', 'protocol', 'port',
+            'protocols', 'is_active', 'public_ip',
+            'number', 'vendor', 'model', 'sn',
+            'cpu_model', 'cpu_count', 'cpu_cores', 'cpu_vcpus', 'memory',
+            'disk_total', 'disk_info', 'os', 'os_version', 'os_arch',
+            'hostname_raw', 'comment', 'created_by', 'date_created',
+            'hardware_info', 'connectivity',
+        ]
+
+    @classmethod
+    def setup_eager_loading(cls, queryset):
+        """ Perform necessary eager loading of data. """
+        queryset = queryset\
+            .annotate(admin_user_username=F('admin_user__username'))
+        return queryset
+
+
 class PlatformSerializer(serializers.ModelSerializer):
     meta = serializers.DictField(required=False, allow_null=True)
 
@@ -151,3 +173,12 @@ class AssetSimpleSerializer(serializers.ModelSerializer):
     class Meta:
         model = Asset
         fields = ['id', 'hostname', 'ip', 'connectivity', 'port']
+
+
+class AssetTaskSerializer(serializers.Serializer):
+    ACTION_CHOICES = (
+        ('refresh', 'refresh'),
+        ('test', 'test'),
+    )
+    task = serializers.CharField(read_only=True)
+    action = serializers.ChoiceField(choices=ACTION_CHOICES, write_only=True)

apps/assets/serializers/asset_user.py

@@ -8,39 +8,23 @@ from common.serializers import AdaptedBulkListSerializer
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 from ..models import AuthBook, Asset
 from ..backends import AssetUserManager
+
 from .base import ConnectivitySerializer, AuthSerializerMixin
 
 
 __all__ = [
-    'AssetUserSerializer', 'AssetUserAuthInfoSerializer',
-    'AssetUserExportSerializer', 'AssetUserPushSerializer',
+    'AssetUserWriteSerializer', 'AssetUserReadSerializer',
+    'AssetUserAuthInfoSerializer', 'AssetUserPushSerializer',
 ]
 
 
-class BasicAssetSerializer(serializers.ModelSerializer):
-    class Meta:
-        model = Asset
-        fields = ['hostname', 'ip']
-
-
-class AssetUserSerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
-    hostname = serializers.CharField(read_only=True, label=_("Hostname"))
-    ip = serializers.CharField(read_only=True, label=_("IP"))
-    connectivity = ConnectivitySerializer(read_only=True, label=_("Connectivity"))
-
-    backend = serializers.CharField(read_only=True, label=_("Backend"))
-
+class AssetUserWriteSerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
     class Meta:
         model = AuthBook
         list_serializer_class = AdaptedBulkListSerializer
-        read_only_fields = (
-            'date_created', 'date_updated', 'created_by',
-            'is_latest', 'version', 'connectivity',
-        )
         fields = [
-            "id", "hostname", "ip", "username", "password", "asset", "version",
-            "is_latest", "connectivity", "backend",
-            "date_created", "date_updated", "private_key", "public_key",
+            'id', 'username', 'password', 'private_key', "public_key",
+            'asset', 'comment',
         ]
         extra_kwargs = {
             'username': {'required': True},
@@ -57,7 +41,32 @@ class AssetUserSerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
         return instance
 
 
-class AssetUserExportSerializer(AssetUserSerializer):
+class AssetUserReadSerializer(AssetUserWriteSerializer):
+    id = serializers.CharField(read_only=True, source='union_id', label=_("ID"))
+    hostname = serializers.CharField(read_only=True, label=_("Hostname"))
+    ip = serializers.CharField(read_only=True, label=_("IP"))
+    asset = serializers.CharField(source='asset_id', label=_('Asset'))
+    backend = serializers.CharField(read_only=True, label=_("Backend"))
+
+    class Meta(AssetUserWriteSerializer.Meta):
+        read_only_fields = (
+            'date_created', 'date_updated',
+            'created_by', 'version',
+        )
+        fields = [
+            'id', 'username', 'password', 'private_key', "public_key",
+            'asset', 'hostname', 'ip', 'backend', 'version',
+            'date_created', "date_updated", 'comment',
+        ]
+        extra_kwargs = {
+            'username': {'required': True},
+            'password': {'write_only': True},
+            'private_key': {'write_only': True},
+            'public_key': {'write_only': True},
+        }
+
+
+class AssetUserAuthInfoSerializer(AssetUserReadSerializer):
     password = serializers.CharField(
         max_length=256, allow_blank=True, allow_null=True,
         required=False, label=_('Password')
@@ -72,12 +81,6 @@ class AssetUserExportSerializer(AssetUserSerializer):
     )
 
 
-class AssetUserAuthInfoSerializer(serializers.ModelSerializer):
-    class Meta:
-        model = AuthBook
-        fields = ['password', 'private_key', 'public_key']
-
-
 class AssetUserPushSerializer(serializers.Serializer):
     asset = serializers.PrimaryKeyRelatedField(queryset=Asset.objects, label=_("Asset"))
     username = serializers.CharField(max_length=1024)

apps/assets/serializers/base.py

@@ -5,6 +5,7 @@ from django.utils.translation import ugettext as _
 from rest_framework import serializers
 
 from common.utils import ssh_pubkey_gen, validate_ssh_private_key
+from ..models import AssetUser
 
 
 class AuthSerializer(serializers.ModelSerializer):
@@ -60,9 +61,6 @@ class AuthSerializerMixin:
             if not value:
                 validated_data.pop(field, None)
 
-        # print(validated_data)
-        # raise serializers.ValidationError(">>>>>>")
-
     def create(self, validated_data):
         self.clean_auth_fields(validated_data)
         return super().create(validated_data)
@@ -70,3 +68,15 @@ class AuthSerializerMixin:
     def update(self, instance, validated_data):
         self.clean_auth_fields(validated_data)
         return super().update(instance, validated_data)
+
+
+class AuthInfoSerializer(serializers.ModelSerializer):
+    private_key = serializers.ReadOnlyField(source='get_private_key')
+
+    class Meta:
+        model = AssetUser
+        fields = [
+            'username', 'password',
+            'private_key', 'public_key',
+            'date_updated',
+        ]

apps/assets/serializers/node.py

@@ -8,7 +8,7 @@ from ..models import Asset, Node
 
 __all__ = [
     'NodeSerializer', "NodeAddChildrenSerializer",
-    "NodeAssetsSerializer",
+    "NodeAssetsSerializer", "NodeTaskSerializer",
 ]
 
 
@@ -51,3 +51,12 @@ class NodeAssetsSerializer(BulkOrgResourceModelSerializer):
 class NodeAddChildrenSerializer(serializers.Serializer):
     nodes = serializers.ListField()
 
+
+class NodeTaskSerializer(serializers.Serializer):
+    ACTION_CHOICES = (
+        ('refresh', 'refresh'),
+        ('test', 'test'),
+        ('refresh_cache', 'refresh_cache'),
+    )
+    task = serializers.CharField(read_only=True)
+    action = serializers.ChoiceField(choices=ACTION_CHOICES, write_only=True)

apps/assets/serializers/system_user.py

@@ -1,20 +1,21 @@
-import re
 from rest_framework import serializers
 
 from django.utils.translation import ugettext_lazy as _
+from django.db.models import Count
 
 from common.serializers import AdaptedBulkListSerializer
 from common.mixins.serializers import BulkSerializerMixin
 from common.utils import ssh_pubkey_gen
 from orgs.mixins.serializers import BulkOrgResourceModelSerializer
 from assets.models import Node
-from ..models import SystemUser
-from .base import AuthSerializer, AuthSerializerMixin
+from ..models import SystemUser, Asset
+from .base import AuthSerializerMixin
 
 __all__ = [
-    'SystemUserSerializer', 'SystemUserAuthSerializer',
+    'SystemUserSerializer', 'SystemUserListSerializer',
     'SystemUserSimpleSerializer', 'SystemUserAssetRelationSerializer',
-    'SystemUserNodeRelationSerializer',
+    'SystemUserNodeRelationSerializer', 'SystemUserTaskSerializer',
+    'SystemUserUserRelationSerializer', 'SystemUserWithAuthInfoSerializer',
 ]
 
 
@@ -28,10 +29,13 @@ class SystemUserSerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
         model = SystemUser
         list_serializer_class = AdaptedBulkListSerializer
         fields = [
-            'id', 'name', 'username', 'password', 'public_key', 'private_key',
-            'login_mode', 'login_mode_display', 'priority', 'protocol',
+            'id', 'name', 'username', 'protocol',
+            'password', 'public_key', 'private_key',
+            'login_mode', 'login_mode_display',
+            'priority', 'username_same_with_user',
             'auto_push', 'cmd_filters', 'sudo', 'shell', 'comment',
-            'assets_amount', 'nodes_amount', 'auto_generate_key'
+            'auto_generate_key', 'sftp_root',
+            'assets_amount',
         ]
         extra_kwargs = {
             'password': {"write_only": True},
@@ -67,17 +71,43 @@ class SystemUserSerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
             value = False
         return value
 
+    def validate_username_same_with_user(self, username_same_with_user):
+        if not username_same_with_user:
+            return username_same_with_user
+        protocol = self.initial_data.get("protocol", "ssh")
+        queryset = SystemUser.objects.filter(
+                protocol=protocol, username_same_with_user=True
+        )
+        if self.instance:
+            queryset = queryset.exclude(id=self.instance.id)
+        exists = queryset.exists()
+        if not exists:
+            return username_same_with_user
+        error = _("Username same with user with protocol {} only allow 1").format(protocol)
+        raise serializers.ValidationError(error)
+
     def validate_username(self, username):
         if username:
             return username
         login_mode = self.initial_data.get("login_mode")
         protocol = self.initial_data.get("protocol")
+        username_same_with_user = self.initial_data.get("username_same_with_user")
+        if username_same_with_user:
+            return ''
         if login_mode == SystemUser.LOGIN_AUTO and \
                 protocol != SystemUser.PROTOCOL_VNC:
             msg = _('* Automatic login mode must fill in the username.')
             raise serializers.ValidationError(msg)
         return username
 
+    def validate_sftp_root(self, value):
+        if value in ['home', 'tmp']:
+            return value
+        if not value.startswith('/'):
+            error = _("Path should starts with /")
+            raise serializers.ValidationError(error)
+        return value
+
     def validate_password(self, password):
         super().validate_password(password)
         auto_gen_key = self.initial_data.get("auto_generate_key", False)
@@ -112,29 +142,34 @@ class SystemUserSerializer(AuthSerializerMixin, BulkOrgResourceModelSerializer):
             attrs["public_key"] = public_key
         return attrs
 
+
+class SystemUserListSerializer(SystemUserSerializer):
+    class Meta(SystemUserSerializer.Meta):
+        fields = [
+            'id', 'name', 'username', 'protocol',
+            'login_mode', 'login_mode_display',
+            'priority', "username_same_with_user",
+            'auto_push', 'sudo', 'shell', 'comment',
+            "assets_amount",
+            'auto_generate_key',
+            'sftp_root',
+        ]
+
     @classmethod
     def setup_eager_loading(cls, queryset):
         """ Perform necessary eager loading of data. """
-        queryset = queryset.prefetch_related('cmd_filters', 'nodes')
+        queryset = queryset.annotate(assets_amount=Count("assets"))
         return queryset
 
 
-class SystemUserAuthSerializer(AuthSerializer):
-    """
-    系统用户认证信息
-    """
-    private_key = serializers.SerializerMethodField()
-
-    class Meta:
-        model = SystemUser
-        fields = [
-            "id", "name", "username", "protocol",
-            "login_mode", "password", "private_key",
-        ]
-
-    @staticmethod
-    def get_private_key(obj):
-        return obj.get_private_key()
+class SystemUserWithAuthInfoSerializer(SystemUserSerializer):
+    class Meta(SystemUserSerializer.Meta):
+        extra_kwargs = {
+            'nodes_amount': {'label': _('Node')},
+            'assets_amount': {'label': _('Asset')},
+            'login_mode_display': {'label': _('Login mode display')},
+            'created_by': {'read_only': True},
+        }
 
 
 class SystemUserSimpleSerializer(serializers.ModelSerializer):
@@ -186,3 +221,25 @@ class SystemUserNodeRelationSerializer(RelationMixin, serializers.ModelSerialize
             return self.tree.get_node_full_tag(obj.node_key)
         else:
             return obj.node.full_value
+
+
+class SystemUserUserRelationSerializer(RelationMixin, serializers.ModelSerializer):
+    user_display = serializers.ReadOnlyField()
+
+    class Meta(RelationMixin.Meta):
+        model = SystemUser.users.through
+        fields = [
+            'id', "user", "user_display",
+        ]
+
+
+class SystemUserTaskSerializer(serializers.Serializer):
+    ACTION_CHOICES = (
+        ("test", "test"),
+        ("push", "push"),
+    )
+    action = serializers.ChoiceField(choices=ACTION_CHOICES, write_only=True)
+    asset = serializers.PrimaryKeyRelatedField(
+        queryset=Asset.objects, allow_null=True, required=False, write_only=True
+    )
+    task = serializers.CharField(read_only=True)

apps/assets/signals_handler.py

@@ -7,8 +7,9 @@ from django.db.models.signals import (
 from django.db.models.aggregates import Count
 from django.dispatch import receiver
 
-from common.utils import get_logger, timeit
+from common.utils import get_logger
 from common.decorator import on_transaction_commit
+from orgs.utils import tmp_to_root_org
 from .models import Asset, SystemUser, Node, AuthBook
 from .utils import TreeService
 from .tasks import (
@@ -113,6 +114,20 @@ def on_system_user_nodes_change(sender, instance=None, action=None, model=None,
     add_nodes_assets_to_system_users.delay(nodes_keys, system_users)
 
 
+@receiver(m2m_changed, sender=SystemUser.groups.through)
+def on_system_user_groups_change(sender, instance=None, action=None, model=None,
+                                 pk_set=None, reverse=False, **kwargs):
+    """
+    当系统用户和用户组关系发生变化时，应该将组下用户关联到新的系统用户上
+    """
+    if action != "post_add" or reverse:
+        return
+    logger.info("System user groups update signal recv: {}".format(instance))
+    groups = model.objects.filter(pk__in=pk_set).annotate(users_count=Count("users"))
+    users = groups.filter(users_count__gt=0).values_list('users', flat=True)
+    instance.users.add(*tuple(users))
+
+
 @receiver(m2m_changed, sender=Asset.nodes.through)
 def on_asset_nodes_change(sender, instance=None, action='', **kwargs):
     """
@@ -121,6 +136,8 @@ def on_asset_nodes_change(sender, instance=None, action='', **kwargs):
     if action.startswith('post'):
         logger.debug("Asset nodes change signal recv: {}".format(instance))
         Node.refresh_assets()
+        with tmp_to_root_org():
+            Node.refresh_assets()
 
 
 @receiver(m2m_changed, sender=Asset.nodes.through)
@@ -195,6 +212,8 @@ def on_asset_nodes_remove(sender, instance=None, action='', model=None,
 def on_node_update_or_created(sender, **kwargs):
     # 刷新节点
     Node.refresh_nodes()
+    with tmp_to_root_org():
+        Node.refresh_nodes()
 
 
 @receiver(post_save, sender=AuthBook)

apps/assets/tasks/admin_user_connectivity.py

@@ -4,11 +4,12 @@ from celery import shared_task
 from django.utils.translation import ugettext as _
 from django.core.cache import cache
 
+from orgs.utils import tmp_to_root_org, org_aware_func
 from common.utils import get_logger
 from ops.celery.decorator import register_as_period_task
 
 from ..models import AdminUser
-from .utils import clean_hosts
+from .utils import clean_ansible_task_hosts
 from .asset_connectivity import test_asset_connectivity_util
 from . import const
 
@@ -20,7 +21,7 @@ __all__ = [
 ]
 
 
-@shared_task(queue="ansible")
+@org_aware_func("admin_user")
 def test_admin_user_connectivity_util(admin_user, task_name):
     """
     Test asset admin user can connect or not. Using ansible api do that
@@ -29,7 +30,7 @@ def test_admin_user_connectivity_util(admin_user, task_name):
     :return:
     """
     assets = admin_user.get_related_assets()
-    hosts = clean_hosts(assets)
+    hosts = clean_ansible_task_hosts(assets)
     if not hosts:
         return {}
     summary = test_asset_connectivity_util(hosts, task_name)
@@ -51,10 +52,13 @@ def test_admin_user_connectivity_period():
         logger.debug("Test admin user connectivity, less than 40 minutes, skip")
         return
     cache.set(key, 1, 60*40)
-    admin_users = AdminUser.objects.all()
-    for admin_user in admin_users:
-        task_name = _("Test admin user connectivity period: {}").format(admin_user.name)
-        test_admin_user_connectivity_util(admin_user, task_name)
+    with tmp_to_root_org():
+        admin_users = AdminUser.objects.all()
+        for admin_user in admin_users:
+            task_name = _("Test admin user connectivity period: {}").format(
+                admin_user.name
+            )
+            test_admin_user_connectivity_util(admin_user, task_name)
     cache.set(key, 1, 60*40)
 
 

apps/assets/tasks/asset_connectivity.py

@@ -1,55 +1,55 @@
 # ~*~ coding: utf-8 ~*~
+from itertools import groupby
 from collections import defaultdict
 from celery import shared_task
 from django.utils.translation import ugettext as _
 
 from common.utils import get_logger
+from orgs.utils import org_aware_func
 from ..models.utils import Connectivity
 from . import const
-from .utils import clean_hosts
+from .utils import clean_ansible_task_hosts, group_asset_by_platform
 
 
 logger = get_logger(__file__)
-__all__ = ['test_asset_connectivity_util', 'test_asset_connectivity_manual']
+__all__ = [
+    'test_asset_connectivity_util', 'test_asset_connectivity_manual',
+    'test_node_assets_connectivity_manual',
+]
 
 
 @shared_task(queue="ansible")
+@org_aware_func("assets")
 def test_asset_connectivity_util(assets, task_name=None):
     from ops.utils import update_or_create_ansible_task
 
     if task_name is None:
         task_name = _("Test assets connectivity")
 
-    hosts = clean_hosts(assets)
+    hosts = clean_ansible_task_hosts(assets)
     if not hosts:
         return {}
+    platform_hosts_map = {}
+    hosts_sorted = sorted(hosts, key=group_asset_by_platform)
+    platform_hosts = groupby(hosts_sorted, key=group_asset_by_platform)
+    for i in platform_hosts:
+        platform_hosts_map[i[0]] = list(i[1])
 
-    hosts_category = {
-        'linux': {
-            'hosts': [],
-            'tasks': const.TEST_ADMIN_USER_CONN_TASKS
-        },
-        'windows': {
-            'hosts': [],
-            'tasks': const.TEST_WINDOWS_ADMIN_USER_CONN_TASKS
-        }
+    platform_tasks_map = {
+        "unixlike": const.PING_UNIXLIKE_TASKS,
+        "windows": const.PING_WINDOWS_TASKS
     }
-    for host in hosts:
-        hosts_list = hosts_category['windows']['hosts'] if host.is_windows() \
-            else hosts_category['linux']['hosts']
-        hosts_list.append(host)
-
     results_summary = dict(
         contacted=defaultdict(dict), dark=defaultdict(dict), success=True
     )
-    created_by = assets[0].org_id
-    for k, value in hosts_category.items():
-        if not value['hosts']:
+    for platform, _hosts in platform_hosts_map.items():
+        if not _hosts:
             continue
+        logger.debug("System user not has special auth")
+        tasks = platform_tasks_map.get(platform)
         task, created = update_or_create_ansible_task(
-            task_name=task_name, hosts=value['hosts'], tasks=value['tasks'],
+            task_name=task_name, hosts=_hosts, tasks=tasks,
             pattern='all', options=const.TASK_OPTIONS, run_as_admin=True,
-            created_by=created_by,
         )
         raw, summary = task.run()
         success = summary.get('success', False)
@@ -59,6 +59,7 @@ def test_asset_connectivity_util(assets, task_name=None):
         results_summary['success'] &= success
         results_summary['contacted'].update(contacted)
         results_summary['dark'].update(dark)
+        continue
 
     for asset in assets:
         if asset.hostname in results_summary.get('dark', {}).keys():
@@ -79,3 +80,12 @@ def test_asset_connectivity_manual(asset):
         return False, summary['dark']
     else:
         return True, ""
+
+
+@shared_task(queue="ansible")
+def test_node_assets_connectivity_manual(node):
+    task_name = _("Test if the assets under the node are connectable: {}".format(node.name))
+    assets = node.get_all_assets()
+    result = test_asset_connectivity_util(assets, task_name=task_name)
+    return result
+

apps/assets/tasks/asset_user_connectivity.py

@@ -3,7 +3,9 @@
 from celery import shared_task
 from django.utils.translation import ugettext as _
 
-from common.utils import get_logger
+from common.utils import get_logger, get_object_or_none
+from orgs.utils import org_aware_func
+from ..models import Asset
 from . import const
 from .utils import check_asset_can_run_ansible
 
@@ -13,15 +15,16 @@ logger = get_logger(__file__)
 
 __all__ = [
     'test_asset_user_connectivity_util', 'test_asset_users_connectivity_manual',
-    'get_test_asset_user_connectivity_tasks',
+    'get_test_asset_user_connectivity_tasks', 'test_user_connectivity',
+    'run_adhoc',
 ]
 
 
 def get_test_asset_user_connectivity_tasks(asset):
     if asset.is_unixlike():
-        tasks = const.TEST_ASSET_USER_CONN_TASKS
+        tasks = const.PING_UNIXLIKE_TASKS
     elif asset.is_windows():
-        tasks = const.TEST_WINDOWS_ASSET_USER_CONN_TASKS
+        tasks = const.PING_WINDOWS_TASKS
     else:
         msg = _(
             "The asset {} system platform {} does not "
@@ -32,46 +35,98 @@ def get_test_asset_user_connectivity_tasks(asset):
     return tasks
 
 
-@shared_task(queue="ansible")
-def test_asset_user_connectivity_util(asset_user, task_name, run_as_admin=False):
+def run_adhoc(task_name, tasks, inventory):
+    """
+    :param task_name
+    :param tasks
+    :param inventory
+    """
+    from ops.ansible.runner import AdHocRunner
+    runner = AdHocRunner(inventory, options=const.TASK_OPTIONS)
+    result = runner.run(tasks, 'all', task_name)
+    return result.results_raw, result.results_summary
+
+
+def test_user_connectivity(task_name, asset, username, password=None, private_key=None):
+    """
+    :param task_name
+    :param asset
+    :param username
+    :param password
+    :param private_key
+    """
+    from ops.inventory import JMSCustomInventory
+
+    tasks = get_test_asset_user_connectivity_tasks(asset)
+    if not tasks:
+        logger.debug("No tasks ")
+        return {}, {}
+    inventory = JMSCustomInventory(
+        assets=[asset], username=username, password=password,
+        private_key=private_key
+    )
+    raw, summary = run_adhoc(
+        task_name=task_name, tasks=tasks, inventory=inventory
+    )
+    return raw, summary
+
+
+@org_aware_func("asset_user")
+def test_asset_user_connectivity_util(asset_user, task_name):
     """
     :param asset_user: <AuthBook>对象
     :param task_name:
-    :param run_as_admin:
     :return:
     """
-    from ops.utils import update_or_create_ansible_task
-
     if not check_asset_can_run_ansible(asset_user.asset):
         return
 
-    tasks = get_test_asset_user_connectivity_tasks(asset_user.asset)
-    if not tasks:
-        logger.debug("No tasks ")
+    try:
+        raw, summary = test_user_connectivity(
+            task_name=task_name, asset=asset_user.asset,
+            username=asset_user.username, password=asset_user.password,
+            private_key=asset_user.private_key
+        )
+    except Exception as e:
+        logger.warn("Failed run adhoc {}, {}".format(task_name, e))
         return
-
-    args = (task_name,)
-    kwargs = {
-        'hosts': [asset_user.asset], 'tasks': tasks,
-        'pattern': 'all', 'options': const.TASK_OPTIONS,
-        'created_by': asset_user.org_id,
-    }
-    if run_as_admin:
-        kwargs["run_as_admin"] = True
-    else:
-        kwargs["run_as"] = asset_user.username
-    task, created = update_or_create_ansible_task(*args, **kwargs)
-    raw, summary = task.run()
     asset_user.set_connectivity(summary)
 
 
 @shared_task(queue="ansible")
-def test_asset_users_connectivity_manual(asset_users, run_as_admin=False):
+def test_asset_users_connectivity_manual(asset_users):
     """
     :param asset_users: <AuthBook>对象
     """
     for asset_user in asset_users:
         task_name = _("Test asset user connectivity: {}").format(asset_user)
-        test_asset_user_connectivity_util(asset_user, task_name, run_as_admin=run_as_admin)
+        test_asset_user_connectivity_util(asset_user, task_name)
+
+
+@shared_task(queue="ansible")
+def push_asset_user_util(asset_user):
+    """
+    :param asset_user: <Asset user>对象
+    """
+    from .push_system_user import push_system_user_util
+    if not asset_user.backend.startswith('system_user'):
+        logger.error("Asset user is not from system user")
+        return
+    union_id = asset_user.union_id
+    union_id_list = union_id.split('_')
+    if len(union_id_list) < 2:
+        logger.error("Asset user union id length less than 2")
+        return
+    system_user_id = union_id_list[0]
+    asset_id = union_id_list[1]
+    asset = get_object_or_none(Asset, pk=asset_id)
+    system_user = None
+    if not asset:
+        return
+    hosts = check_asset_can_run_ansible([asset])
+    if asset.is_unixlike:
+        pass
+
+
 
 

apps/assets/tasks/const.py

@@ -18,27 +18,10 @@ UPDATE_ASSETS_HARDWARE_TASKS = [
    }
 ]
 
-TEST_ADMIN_USER_CONN_TASKS = [
-    {
-        "name": "ping",
-        "action": {
-            "module": "ping",
-        }
-    }
-]
-TEST_WINDOWS_ADMIN_USER_CONN_TASKS = [
-    {
-        "name": "ping",
-        "action": {
-            "module": "win_ping",
-        }
-    }
-]
-
 ASSET_ADMIN_CONN_CACHE_KEY = "ASSET_ADMIN_USER_CONN_{}"
 
 SYSTEM_USER_CONN_CACHE_KEY = "SYSTEM_USER_CONN_{}"
-TEST_SYSTEM_USER_CONN_TASKS = [
+PING_UNIXLIKE_TASKS = [
    {
        "name": "ping",
        "action": {
@@ -46,7 +29,7 @@ TEST_SYSTEM_USER_CONN_TASKS = [
        }
    }
 ]
-TEST_WINDOWS_SYSTEM_USER_CONN_TASKS = [
+PING_WINDOWS_TASKS = [
     {
         "name": "ping",
         "action": {
@@ -55,24 +38,6 @@ TEST_WINDOWS_SYSTEM_USER_CONN_TASKS = [
     }
 ]
 
-TEST_ASSET_USER_CONN_TASKS = [
-    {
-        "name": "ping",
-        "action": {
-            "module": "ping",
-        }
-    }
-]
-TEST_WINDOWS_ASSET_USER_CONN_TASKS = [
-    {
-        "name": "ping",
-        "action": {
-            "module": "win_ping",
-        }
-    }
-]
-
-
 TASK_OPTIONS = {
     'timeout': 10,
     'forks': 10,
@@ -98,7 +63,9 @@ GATHER_ASSET_USERS_TASKS = [
         "name": "get last login",
         "action": {
             "module": "shell",
-            "args": "users=$(getent passwd | grep -v 'nologin' | grep -v 'shudown' | awk -F: '{ print $1 }');for i in $users;do last -F $i -1 |  head -1 | grep -v '^$'  | awk '{ print $1\"@\"$3\"@\"$5,$6,$7,$8 }';done"
+            "args": "users=$(getent passwd | grep -v 'nologin' | "
+                    "grep -v 'shudown' | awk -F: '{ print $1 }');for i in $users;do last -F $i -1 | "
+                    "head -1 | grep -v '^$'  | awk '{ print $1\"@\"$3\"@\"$5,$6,$7,$8 }';done"
         }
     }
 ]

apps/assets/tasks/gather_asset_hardware_info.py

@@ -9,15 +9,16 @@ from django.utils.translation import ugettext as _
 from common.utils import (
     capacity_convert, sum_capacity, get_logger
 )
+from orgs.utils import org_aware_func
 from . import const
-from .utils import clean_hosts
+from .utils import clean_ansible_task_hosts
 
 
 logger = get_logger(__file__)
 disk_pattern = re.compile(r'^hd|sd|xvd|vd|nv')
 __all__ = [
     'update_assets_hardware_info_util', 'update_asset_hardware_info_manual',
-    'update_assets_hardware_info_period',
+    'update_assets_hardware_info_period',  'update_node_assets_hardware_info_manual',
 ]
 
 
@@ -82,6 +83,7 @@ def set_assets_hardware_info(assets, result, **kwargs):
 
 
 @shared_task
+@org_aware_func("assets")
 def update_assets_hardware_info_util(assets, task_name=None):
     """
     Using ansible api to update asset hardware info
@@ -93,13 +95,13 @@ def update_assets_hardware_info_util(assets, task_name=None):
     if task_name is None:
         task_name = _("Update some assets hardware info")
     tasks = const.UPDATE_ASSETS_HARDWARE_TASKS
-    hosts = clean_hosts(assets)
+    hosts = clean_ansible_task_hosts(assets)
     if not hosts:
         return {}
-    created_by = str(assets[0].org_id)
     task, created = update_or_create_ansible_task(
-        task_name, hosts=hosts, tasks=tasks, created_by=created_by,
-        pattern='all', options=const.TASK_OPTIONS, run_as_admin=True,
+        task_name, hosts=hosts, tasks=tasks,
+        pattern='all', options=const.TASK_OPTIONS,
+        run_as_admin=True,
     )
     result = task.run()
     set_assets_hardware_info(assets, result)
@@ -109,9 +111,7 @@ def update_assets_hardware_info_util(assets, task_name=None):
 @shared_task(queue="ansible")
 def update_asset_hardware_info_manual(asset):
     task_name = _("Update asset hardware info: {}").format(asset.hostname)
-    update_assets_hardware_info_util(
-        [asset], task_name=task_name
-    )
+    update_assets_hardware_info_util([asset], task_name=task_name)
 
 
 @shared_task(queue="ansible")
@@ -123,3 +123,11 @@ def update_assets_hardware_info_period():
     if not const.PERIOD_TASK_ENABLED:
         logger.debug("Period task disabled, update assets hardware info pass")
         return
+
+
+@shared_task(queue="ansible")
+def update_node_assets_hardware_info_manual(node):
+    task_name = _("Update node asset hardware information: {}").format(node.name)
+    assets = node.get_all_assets()
+    result = update_assets_hardware_info_util.delay(assets, task_name=task_name)
+    return result

apps/assets/tasks/gather_asset_users.py

@@ -7,10 +7,10 @@ from celery import shared_task
 from django.utils.translation import ugettext as _
 from django.utils import timezone
 
-from orgs.utils import tmp_to_org
+from orgs.utils import tmp_to_org, org_aware_func
 from common.utils import get_logger
 from ..models import GatheredUser, Node
-from .utils import clean_hosts
+from .utils import clean_ansible_task_hosts
 from . import const
 
 __all__ = ['gather_asset_users', 'gather_nodes_asset_users']
@@ -101,11 +101,12 @@ def add_asset_users(assets, results):
 
 
 @shared_task(queue="ansible")
+@org_aware_func("assets")
 def gather_asset_users(assets, task_name=None):
     from ops.utils import update_or_create_ansible_task
     if task_name is None:
         task_name = _("Gather assets users")
-    assets = clean_hosts(assets)
+    assets = clean_ansible_task_hosts(assets)
     if not assets:
         return
     hosts_category = {
@@ -131,7 +132,7 @@ def gather_asset_users(assets, task_name=None):
         task, created = update_or_create_ansible_task(
             task_name=_task_name, hosts=value['hosts'], tasks=value['tasks'],
             pattern='all', options=const.TASK_OPTIONS,
-            run_as_admin=True, created_by=value['hosts'][0].org_id,
+            run_as_admin=True,
         )
         raw, summary = task.run()
         results[k].update(raw['ok'])

apps/assets/tasks/push_system_user.py

@@ -1,11 +1,13 @@
 # ~*~ coding: utf-8 ~*~
 
+from itertools import groupby
 from celery import shared_task
 from django.utils.translation import ugettext as _
 
 from common.utils import encrypt_password, get_logger
+from orgs.utils import tmp_to_org, org_aware_func
 from . import const
-from .utils import clean_hosts_by_protocol, clean_hosts
+from .utils import clean_ansible_task_hosts, group_asset_by_platform
 
 
 logger = get_logger(__file__)
@@ -15,31 +17,34 @@ __all__ = [
 ]
 
 
-def get_push_linux_system_user_tasks(system_user):
+def get_push_unixlike_system_user_tasks(system_user, username=None):
+    if username is None:
+        username = system_user.username
+    password = system_user.password
+    public_key = system_user.public_key
+
     tasks = [
         {
-            'name': 'Add user {}'.format(system_user.username),
+            'name': 'Add user {}'.format(username),
             'action': {
                 'module': 'user',
                 'args': 'name={} shell={} state=present'.format(
-                    system_user.username, system_user.shell,
+                    username, system_user.shell or '/bin/bash',
                 ),
             }
         },
         {
-            'name': 'Add group {}'.format(system_user.username),
+            'name': 'Add group {}'.format(username),
             'action': {
                 'module': 'group',
-                'args': 'name={} state=present'.format(
-                    system_user.username,
-                ),
+                'args': 'name={} state=present'.format(username),
             }
         },
         {
             'name': 'Check home dir exists',
             'action': {
                 'module': 'stat',
-                'args': 'path=/home/{}'.format(system_user.username)
+                'args': 'path=/home/{}'.format(username)
             },
             'register': 'home_existed'
         },
@@ -47,29 +52,29 @@ def get_push_linux_system_user_tasks(system_user):
             'name': "Set home dir permission",
             'action': {
                 'module': 'file',
-                'args': "path=/home/{0} owner={0} group={0} mode=700".format(system_user.username)
+                'args': "path=/home/{0} owner={0} group={0} mode=700".format(username)
             },
             'when': 'home_existed.stat.exists == true'
         }
     ]
-    if system_user.password:
+    if password:
         tasks.append({
-            'name': 'Set {} password'.format(system_user.username),
+            'name': 'Set {} password'.format(username),
             'action': {
                 'module': 'user',
                 'args': 'name={} shell={} state=present password={}'.format(
-                    system_user.username, system_user.shell,
-                    encrypt_password(system_user.password, salt="K3mIlKK"),
+                    username, system_user.shell,
+                    encrypt_password(password, salt="K3mIlKK"),
                 ),
             }
         })
-    if system_user.public_key:
+    if public_key:
         tasks.append({
-            'name': 'Set {} authorized key'.format(system_user.username),
+            'name': 'Set {} authorized key'.format(username),
             'action': {
                 'module': 'authorized_key',
                 'args': "user={} state=present key='{}'".format(
-                    system_user.username, system_user.public_key
+                    username, public_key
                 )
             }
         })
@@ -81,26 +86,27 @@ def get_push_linux_system_user_tasks(system_user):
             sudo_tmp.append(s.strip(','))
         sudo = ','.join(sudo_tmp)
         tasks.append({
-            'name': 'Set {} sudo setting'.format(system_user.username),
+            'name': 'Set {} sudo setting'.format(username),
             'action': {
                 'module': 'lineinfile',
                 'args': "dest=/etc/sudoers state=present regexp='^{0} ALL=' "
                         "line='{0} ALL=(ALL) NOPASSWD: {1}' "
-                        "validate='visudo -cf %s'".format(
-                    system_user.username, sudo,
-                )
+                        "validate='visudo -cf %s'".format(username, sudo)
             }
         })
 
     return tasks
 
 
-def get_push_windows_system_user_tasks(system_user):
+def get_push_windows_system_user_tasks(system_user, username=None):
+    if username is None:
+        username = system_user.username
+    password = system_user.password
     tasks = []
-    if not system_user.password:
+    if not password:
         return tasks
-    tasks.append({
-        'name': 'Add user {}'.format(system_user.username),
+    task = {
+        'name': 'Add user {}'.format(username),
         'action': {
             'module': 'win_user',
             'args': 'fullname={} '
@@ -112,78 +118,95 @@ def get_push_windows_system_user_tasks(system_user):
                     'password_never_expires=yes '
                     'groups="Users,Remote Desktop Users" '
                     'groups_action=add '
-                    ''.format(system_user.name,
-                              system_user.username,
-                              system_user.password),
+                    ''.format(username, username, password),
         }
-    })
+    }
+    print(task)
+    tasks.append(task)
     return tasks
 
 
-def get_push_system_user_tasks(host, system_user):
-    if host.is_unixlike():
-        tasks = get_push_linux_system_user_tasks(system_user)
-    elif host.is_windows():
-        tasks = get_push_windows_system_user_tasks(system_user)
-    else:
-        msg = _(
-            "The asset {} system platform {} does not "
-            "support run Ansible tasks".format(host.hostname, host.platform)
-        )
-        logger.info(msg)
-        tasks = []
+def get_push_system_user_tasks(system_user, platform="unixlike", username=None):
+    """
+    :param system_user:
+    :param platform:
+    :param username: 当动态时，近推送某个
+    :return:
+    """
+    get_task_map = {
+        "unixlike": get_push_unixlike_system_user_tasks,
+        "windows": get_push_windows_system_user_tasks,
+    }
+    get_tasks = get_task_map.get(platform, get_push_unixlike_system_user_tasks)
+    if not system_user.username_same_with_user:
+        return get_tasks(system_user)
+    tasks = []
+    # 仅推送这个username
+    if username is not None:
+        tasks.extend(get_tasks(system_user, username))
+        return tasks
+    users = system_user.users.all().values_list('username', flat=True)
+    print(_("System user is dynamic: {}").format(list(users)))
+    for _username in users:
+        tasks.extend(get_tasks(system_user, _username))
     return tasks
 
 
-@shared_task(queue="ansible")
-def push_system_user_util(system_user, assets, task_name):
+@org_aware_func("system_user")
+def push_system_user_util(system_user, assets, task_name, username=None):
     from ops.utils import update_or_create_ansible_task
-    if not system_user.is_need_push():
-        msg = _("Push system user task skip, auto push not enable or "
-                "protocol is not ssh or rdp: {}").format(system_user.name)
-        logger.info(msg)
-        return {}
-
-    # Set root as system user is dangerous
-    if system_user.username.lower() in ["root", "administrator"]:
-        msg = _("For security, do not push user {}".format(system_user.username))
-        logger.info(msg)
-        return {}
-
-    hosts = clean_hosts(assets)
+    hosts = clean_ansible_task_hosts(assets, system_user=system_user)
     if not hosts:
         return {}
 
-    hosts = clean_hosts_by_protocol(system_user, hosts)
-    if not hosts:
-        return {}
+    platform_hosts_map = {}
+    hosts_sorted = sorted(hosts, key=group_asset_by_platform)
+    platform_hosts = groupby(hosts_sorted, key=group_asset_by_platform)
+    for i in platform_hosts:
+        platform_hosts_map[i[0]] = list(i[1])
 
-    for host in hosts:
-        system_user.load_specific_asset_auth(host)
-        tasks = get_push_system_user_tasks(host, system_user)
-        if not tasks:
-            continue
+    def run_task(_tasks, _hosts):
+        if not _tasks:
+            return
         task, created = update_or_create_ansible_task(
-            task_name=task_name, hosts=[host], tasks=tasks, pattern='all',
+            task_name=task_name, hosts=_hosts, tasks=_tasks, pattern='all',
             options=const.TASK_OPTIONS, run_as_admin=True,
-            created_by=system_user.org_id,
         )
         task.run()
 
+    for platform, _hosts in platform_hosts_map.items():
+        if not _hosts:
+            continue
+        print(_("Start push system user for platform: [{}]").format(platform))
+        print(_("Hosts count: {}").format(len(_hosts)))
+
+        if not system_user.has_special_auth():
+            logger.debug("System user not has special auth")
+            tasks = get_push_system_user_tasks(system_user, platform, username=username)
+            run_task(tasks, _hosts)
+            continue
+
+        for _host in _hosts:
+            system_user.load_asset_special_auth(_host)
+            tasks = get_push_system_user_tasks(system_user, platform, username=username)
+            run_task(tasks, [_host])
+
 
 @shared_task(queue="ansible")
 def push_system_user_to_assets_manual(system_user):
-    assets = system_user.get_all_assets()
+    assets = system_user.get_related_assets()
     task_name = _("Push system users to assets: {}").format(system_user.name)
     return push_system_user_util(system_user, assets, task_name=task_name)
 
 
 @shared_task(queue="ansible")
-def push_system_user_a_asset_manual(system_user, asset):
-    task_name = _("Push system users to asset: {} => {}").format(
-        system_user.name, asset
+def push_system_user_a_asset_manual(system_user, asset, username=None):
+    if username is None:
+        username = system_user.username
+    task_name = _("Push system users to asset: {}({}) => {}").format(
+        system_user.name, username, asset
     )
-    return push_system_user_util(system_user, [asset], task_name=task_name)
+    return push_system_user_util(system_user, [asset], task_name=task_name, username=username)
 
 
 @shared_task(queue="ansible")
@@ -199,4 +222,4 @@ def push_system_user_to_assets(system_user, assets):
 # @after_app_shutdown_clean_periodic
 # def push_system_user_period():
 #     for system_user in SystemUser.objects.all():
-#         push_system_user_related_nodes(system_user)
+#         push_system_user_related_nodes(system_user)

apps/assets/tasks/system_user_connectivity.py

@@ -1,13 +1,17 @@
 
+from itertools import groupby
 from collections import defaultdict
+
 from celery import shared_task
 from django.utils.translation import ugettext as _
 
 from common.utils import get_logger
-
+from orgs.utils import tmp_to_org, org_aware_func
 from ..models import SystemUser
 from . import const
-from .utils import clean_hosts, clean_hosts_by_protocol
+from .utils import (
+    clean_ansible_task_hosts, group_asset_by_platform
+)
 
 logger = get_logger(__name__)
 __all__ = [
@@ -16,7 +20,7 @@ __all__ = [
 ]
 
 
-@shared_task(queue="ansible")
+@org_aware_func("system_user")
 def test_system_user_connectivity_util(system_user, assets, task_name):
     """
     Test system cant connect his assets or not.
@@ -27,41 +31,34 @@ def test_system_user_connectivity_util(system_user, assets, task_name):
     """
     from ops.utils import update_or_create_ansible_task
 
-    hosts = clean_hosts(assets)
+    hosts = clean_ansible_task_hosts(assets, system_user=system_user)
     if not hosts:
         return {}
-
-    hosts = clean_hosts_by_protocol(system_user, hosts)
-    if not hosts:
-        return {}
-
-    hosts_category = {
-        'linux': {
-            'hosts': [],
-            'tasks': const.TEST_SYSTEM_USER_CONN_TASKS
-        },
-        'windows': {
-            'hosts': [],
-            'tasks': const.TEST_WINDOWS_SYSTEM_USER_CONN_TASKS
-        }
+    platform_hosts_map = {}
+    hosts_sorted = sorted(hosts, key=group_asset_by_platform)
+    platform_hosts = groupby(hosts_sorted, key=group_asset_by_platform)
+    for i in platform_hosts:
+        platform_hosts_map[i[0]] = list(i[1])
+
+    platform_tasks_map = {
+        "unixlike": const.PING_UNIXLIKE_TASKS,
+        "windows": const.PING_WINDOWS_TASKS
     }
-    for host in hosts:
-        hosts_list = hosts_category['windows']['hosts'] if host.is_windows() \
-            else hosts_category['linux']['hosts']
-        hosts_list.append(host)
 
     results_summary = dict(
         contacted=defaultdict(dict), dark=defaultdict(dict), success=True
     )
-    for k, value in hosts_category.items():
-        if not value['hosts']:
-            continue
-        task, created = update_or_create_ansible_task(
-            task_name=task_name, hosts=value['hosts'], tasks=value['tasks'],
+
+    def run_task(_tasks, _hosts, _username):
+        old_name = "({})".format(system_user.username)
+        new_name = "({})".format(_username)
+        _task_name = task_name.replace(old_name, new_name)
+        _task, created = update_or_create_ansible_task(
+            task_name=_task_name, hosts=_hosts, tasks=_tasks,
             pattern='all', options=const.TASK_OPTIONS,
-            run_as=system_user.username, created_by=system_user.org_id,
+            run_as=_username,
         )
-        raw, summary = task.run()
+        raw, summary = _task.run()
         success = summary.get('success', False)
         contacted = summary.get('contacted', {})
         dark = summary.get('dark', {})
@@ -70,23 +67,45 @@ def test_system_user_connectivity_util(system_user, assets, task_name):
         results_summary['contacted'].update(contacted)
         results_summary['dark'].update(dark)
 
+    for platform, _hosts in platform_hosts_map.items():
+        if not _hosts:
+            continue
+        if platform not in ["unixlike", "windows"]:
+            continue
+
+        tasks = platform_tasks_map[platform]
+        print(_("Start test system user connectivity for platform: [{}]").format(platform))
+        print(_("Hosts count: {}").format(len(_hosts)))
+        # 用户名不是动态的，用户名则是一个
+        if not system_user.username_same_with_user:
+            logger.debug("System user not has special auth")
+            run_task(tasks, _hosts, system_user.username)
+        # 否则需要多个任务
+        else:
+            users = system_user.users.all().values_list('username', flat=True)
+            print(_("System user is dynamic: {}").format(list(users)))
+            for username in users:
+                run_task(tasks, _hosts, username)
+
     system_user.set_connectivity(results_summary)
     return results_summary
 
 
 @shared_task(queue="ansible")
+@org_aware_func("system_user")
 def test_system_user_connectivity_manual(system_user):
     task_name = _("Test system user connectivity: {}").format(system_user)
-    assets = system_user.get_all_assets()
-    return test_system_user_connectivity_util(system_user, assets, task_name)
+    assets = system_user.get_related_assets()
+    test_system_user_connectivity_util(system_user, assets, task_name)
 
 
 @shared_task(queue="ansible")
+@org_aware_func("system_user")
 def test_system_user_connectivity_a_asset(system_user, asset):
     task_name = _("Test system user connectivity: {} => {}").format(
         system_user, asset
     )
-    return test_system_user_connectivity_util(system_user, [asset], task_name)
+    test_system_user_connectivity_util(system_user, [asset], task_name)
 
 
 @shared_task(queue="ansible")
@@ -94,8 +113,9 @@ def test_system_user_connectivity_period():
     if not const.PERIOD_TASK_ENABLED:
         logger.debug("Period task disabled, test system user connectivity pass")
         return
-    system_users = SystemUser.objects.all()
-    for system_user in system_users:
+    queryset_map = SystemUser.objects.all_group_by_org()
+    for org, system_user in queryset_map.items():
         task_name = _("Test system user connectivity period: {}").format(system_user)
-        assets = system_user.get_all_assets()
-        test_system_user_connectivity_util(system_user, assets, task_name)
+        with tmp_to_org(org):
+            assets = system_user.get_related_assets()
+            test_system_user_connectivity_util(system_user, assets, task_name)

apps/assets/tasks/utils.py

@@ -7,7 +7,8 @@ from common.utils import get_logger
 
 logger = get_logger(__file__)
 __all__ = [
-    'check_asset_can_run_ansible', 'clean_hosts', 'clean_hosts_by_protocol'
+    'check_asset_can_run_ansible', 'clean_ansible_task_hosts',
+    'group_asset_by_platform',
 ]
 
 
@@ -23,23 +24,43 @@ def check_asset_can_run_ansible(asset):
     return True
 
 
-def clean_hosts(assets):
-    clean_assets = []
+def check_system_user_can_run_ansible(system_user):
+    if not system_user.is_need_push():
+        msg = _("Push system user task skip, auto push not enable or "
+                "protocol is not ssh or rdp: {}").format(system_user.name)
+        logger.info(msg)
+        return False
+
+    # Push root as system user is dangerous
+    if system_user.username.lower() in ["root", "administrator"]:
+        msg = _("For security, do not push user {}".format(system_user.username))
+        logger.info(msg)
+        return False
+
+    # if system_user.protocol != "ssh":
+    #     msg = _("System user protocol not ssh: {}".format(system_user))
+    #     logger.info(msg)
+    #     return False
+    return True
+
+
+def clean_ansible_task_hosts(assets, system_user=None):
+    if system_user and not check_system_user_can_run_ansible(system_user):
+        return []
+    cleaned_assets = []
     for asset in assets:
         if not check_asset_can_run_ansible(asset):
             continue
-        clean_assets.append(asset)
-    if not clean_assets:
+        cleaned_assets.append(asset)
+    if not cleaned_assets:
         logger.info(_("No assets matched, stop task"))
-    return clean_assets
+    return cleaned_assets
 
 
-def clean_hosts_by_protocol(system_user, assets):
-    hosts = [
-        asset for asset in assets
-        if asset.has_protocol(system_user.protocol)
-    ]
-    if not hosts:
-        msg = _("No assets matched related system user protocol, stop task")
-        logger.info(msg)
-    return hosts
+def group_asset_by_platform(asset):
+    if asset.is_unixlike():
+        return 'unixlike'
+    elif asset.is_windows():
+        return 'windows'
+    else:
+        return 'other'

apps/assets/templates/assets/_asset_user_auth_update_modal.html

@@ -6,25 +6,25 @@
 <form class="form-horizontal" role="form" onkeydown="if(event.keyCode==13){ $('#btn_asset_user_auth_update_modal_confirm').trigger('click'); return false;}">
     {% csrf_token %}
     <div class="form-group">
-        <label class="col-sm-2 control-label">{% trans "Hostname" %}</label>
+        <label class="col-sm-2 control-label">{% trans "Hostname" %}: </label>
         <div class="col-sm-10">
             <p class="form-control-static" id="id_hostname_p"></p>
         </div>
     </div>
     <div class="form-group">
-        <label class="col-sm-2 control-label">{% trans "Username" %}</label>
+        <label class="col-sm-2 control-label">{% trans "Username" %}: </label>
         <div class="col-sm-10">
             <p class="form-control-static" id="id_username_p"></p>
         </div>
     </div>
     <div class="form-group">
-        <label class="col-sm-2 control-label">{% trans "Password" %}</label>
+        <label class="col-sm-2 control-label">{% trans "Password" %}: </label>
         <div class="col-sm-10">
             <input class="form-control" id="id_password_auth" type="password" name="password" placeholder="{% trans 'Please input password' %}"/>
         </div>
     </div>
     <div class="form-group">
-        <label class="col-sm-2 control-label">{% trans "Private key" %}</label>
+        <label class="col-sm-2 control-label">{% trans "Private key" %}: </label>
         <div class="col-sm-10">
             <div class="row bootstrap3-multi-input">
                 <div class="col-xs-12">

apps/assets/templates/assets/_asset_user_auth_view_modal.html

@@ -12,19 +12,19 @@
 <form class="form-horizontal" action="" style="padding-top: 20px">
 <div class="auth-field">
     <div class="form-group">
-        <label for="" class="col-sm-2 control-label">{% trans 'Hostname' %}</label>
+        <label for="" class="col-sm-2 control-label">{% trans 'Hostname' %}:</label>
         <div class="col-sm-8">
             <p class="form-control-static" id="id_hostname_view"></p>
         </div>
     </div>
     <div class="form-group">
-        <label for="" class="col-sm-2 control-label">{% trans 'Username' %}</label>
+        <label for="" class="col-sm-2 control-label">{% trans 'Username' %}:</label>
         <div class="col-sm-8" >
             <p class="form-control-static" id="id_username_view"></p>
         </div>
     </div>
     <div class="form-group">
-        <label for="" class="col-sm-2 control-label">{% trans 'Password' %}</label>
+        <label for="" class="col-sm-2 control-label">{% trans 'Password' %}:</label>
         <div class="col-sm-8">
             <input id="id_password_view" type="password" class="form-control" value="" readonly style="border: none;padding-left: 0;background-color: #fff;width: 100%">
         </div>
@@ -38,11 +38,11 @@
 <script src="{% static "js/plugins/clipboard/clipboard.min.js" %}"></script>
 <script>
 var showPassword = false;
-
-var authAssetId = "";
 var authHostname = "";
 var authUsername = "";
 var mfaFor = "";
+var authUid = "";
+var authInfoDetailUrl = "{% url "api-assets:asset-user-auth-info-detail" pk=DEFAULT_PK %}";
 
 function initClipboard() {
     var clipboard = new Clipboard('.btn-copy-password', {
@@ -56,12 +56,10 @@ function initClipboard() {
 }
 
 function showAuth() {
-    var url = "{% url "api-assets:asset-user-auth-info" %}?asset_id=" + authAssetId + "&username=" + authUsername;
-    if (prefer) {
-        url = setUrlParam(url, 'prefer', prefer)
-    }
+    var url = authInfoDetailUrl.replace("{{ DEFAULT_PK }}", authUid);
     $("#id_username_view").html(authUsername);
     $("#id_hostname_view").html(authHostname);
+    $("#id_password_view").val('');
     var success = function (data) {
         var password = data.password;
         $("#id_password_view").val(password);
@@ -89,7 +87,13 @@ $(document).ready(function () {
         $("#id_password_view").attr("type", "password")
     }
 }).on("show.bs.modal", "#asset_user_auth_view", function () {
+    showPassword = false;
+    $("#id_password_view").attr("type", "password");
     showAuth();
+}).on("hide.bs.modal", "#asset_user_auth_view", function () {
+    $("#id_username_view").html('');
+    $("#id_hostname_view").html('');
+    $("#id_password_view").val('');
 })
 </script>
 {% endblock %}

apps/assets/templates/assets/_asset_user_list.html

@@ -8,8 +8,8 @@
     table.dataTable tbody tr.selected a {
         color: rgb(103, 106, 108);;
     }
-
 </style>
+
 <table class="table table-striped table-bordered table-hover" id="asset_user_list_table" style="width: 100%">
   <thead>
   <tr>
@@ -20,7 +20,7 @@
       <th class="text-center">{% trans 'IP' %}</th>
       <th class="text-center">{% trans 'Username' %}</th>
       <th class="text-center">{% trans 'Version' %}</th>
-      <th class="text-center">{% trans 'Connectivity'%}</th>
+{#      <th class="text-center">{% trans 'Connectivity'%}</th>#}
       <th class="text-center">{% trans 'Datetime' %}</th>
       <th class="text-center">{% trans 'Action' %}</th>
    </tr>
@@ -33,62 +33,60 @@
 {% include 'authentication/_mfa_confirm_modal.html' %}
 
 <script>
-var assetUserListUrl = "{% url "api-assets:asset-user-list" %}";
+var defaultAssetUserListUrl = "{% url "api-assets:asset-user-list" %}";
+var defaultAssetUserDetail = "{% url "api-assets:asset-user-detail" pk=DEFAULT_PK %}";
 var assetUserTable;
-var needPush = false;
-var prefer = null;
+var defaultNeedPush = false;
 var lastMFATime = "{{ request.session.MFA_VERIFY_TIME }}";
 var testDatetime = "{% trans 'Test datetime: ' %}";
 var mfaVerifyTTL = "{{ SECURITY_MFA_VERIFY_TTL }}";
 var mfaNeedCheck = "{{ SECURITY_VIEW_AUTH_NEED_MFA }}" === "True";
+var onlyLatestEl = "<span style='padding-right:20px'><input type='checkbox' id='only_latest'> {% trans 'Only latest version' %}</span>";
+var onlyLatestChecked = false;
+var systemUserId = "";
 
-function initAssetUserTable() {
+function initAssetUserTable(option) {
+    if (!option) {
+        option = {}
+    }
+    var assetUserListUrl = option.assetUserListUrl || defaultAssetUserListUrl;
+    var needPush = option.needPush === undefined ? defaultNeedPush : option.needPush;
     var options = {
         ele: $('#asset_user_list_table'),
         toggle: true,
         columnDefs: [
             {
                 targets: 5, createdCell: function (td, cellData) {
-                  var innerHtml = "";
-                  if (cellData.status == 1) {
-                      innerHtml = '<i class="fa fa-circle text-navy"></i>'
-                  } else if (cellData.status == 0) {
-                      innerHtml = '<i class="fa fa-circle text-danger"></i>'
-                  } else {
-                      innerHtml = '<i class="fa fa-circle text-warning"></i>'
-                  }
-                  var dateManual = toSafeLocalDateStr(cellData.datetime);
-                  var dataContent = testDatetime + dateManual;
-                  innerHtml = "<a data-toggle='popover' data-content='" + dataContent + "'" + 'data-placement="auto bottom"' + ">" + innerHtml + "</a>";
-                  $(td).html(innerHtml);
-                }
-            },
-            {
-                targets: 6, createdCell: function (td, cellData) {
                     var data = toSafeLocalDateStr(cellData);
                     $(td).html(data);
                 },
             },
             {
-                targets: 7, createdCell: function (td, cellData, rowData) {
-                    var view_btn = '<button class="btn btn-xs btn-primary m-l-xs btn-view-auth" data-user="username123" data-hostname="hostname123" data-asset="asset123">{% trans "View" %}</button>'
-                    var update_btn = '<li><a class="btn-update-auth" data-user="username123" data-hostname="hostname123" data-asset="asset123">{% trans 'Update' %}</a></li>';
-                    var test_btn = '<li><a class="btn-test-auth" data-user="username123" data-hostname="hostname123" data-asset="asset123">{% trans 'Test' %}</a></li>';
-                    var push_btn = '<li><a class="btn-push-auth" data-user="username123" data-hostname="hostname123" data-asset="asset123">{% trans 'Push' %}</a></li>';
-                    if (needPush) {
-                        test_btn += push_btn;
+                targets: 6, createdCell: function (td, cellData, rowData) {
+                    var viewBtn = '<button class="btn btn-xs btn-primary m-l-xs btn-view-auth" DATA>{% trans "View" %}</button>';
+                    var updateBtn = '<li><a class="btn-update-auth" DATA>{% trans 'Update' %}</a></li>';
+                    var testBtn = '<li><a class="btn-test-auth" DATA>{% trans 'Test' %}</a></li>';
+                    var pushBtn = '<li><a class="btn-push-auth" DATA>{% trans 'Push' %}</a></li>';
+                    var delBtn = '<li><a class="btn-del-auth" DATA>{% trans 'Delete' %}</a></li>';
+                    if (!needPush) {
+                        pushBtn = ''
                     }
-                    var actions = '<div class="btn-group">' + view_btn +
+
+                    var data = "data-hostname=hostname123 data-username=username123 data-uid=uid123 data-asset=asset123";
+                    data = data.replaceAll("username123", rowData.username)
+                                     .replaceAll("hostname123", rowData.hostname)
+                                     .replaceAll("uid123", rowData.id)
+                                     .replaceAll("asset123", rowData.asset);
+
+                    var actions = '<div class="btn-group">' + viewBtn +
                         '             <button data-toggle="dropdown" class="btn btn-primary btn-xs dropdown-toggle">' +
                         '               <span class="caret"></span>' +
                         '             </button>' +
                         '                  <ul class="dropdown-menu">' +
-                                           update_btn + test_btn +
+                                           updateBtn + delBtn + testBtn + pushBtn
                         '                  </ul>' +
                         '           </div>';
-                        actions = actions.replaceAll("username123", rowData.username)
-                                         .replaceAll("hostname123", rowData.hostname)
-                                         .replaceAll("asset123", rowData.asset);
+                    actions = actions.replaceAll("DATA", data);
 				    $(td).html(actions);
                 },
                 width: '70px'
@@ -98,21 +96,23 @@ function initAssetUserTable() {
         columns: [
             {data: "id"}, {data: "hostname"}, {data: "ip"},
             {data: "username"}, {data: "version", orderable: false},
-            {data: "connectivity"},
             {data: "date_created", orderable: false},
             {data: "asset", orderable: false}
         ],
-        op_html: $('#actions').html()
+        op_html: $('#actions').html(),
+        lb_html: onlyLatestEl,
     };
-    table = jumpserver.initServerSideDataTable(options);
-    return table
+    assetUserTable = jumpserver.initServerSideDataTable(options);
+    return assetUserTable
 }
 $(document).ready(function(){
 })
 .on('click', '.btn-view-auth', function () {
-    authAssetId = $(this).data("asset") ;
+    // 通知给view auth modal
+    authAssetId = $(this).data("asset");
     authHostname = $(this).data("hostname");
-    authUsername = $(this).data('user');
+    authUsername = $(this).data('username');
+    authUid = $(this).data("uid");
     if (!mfaNeedCheck){
         $("#asset_user_auth_view").modal('show');
         return
@@ -133,29 +133,56 @@ $(document).ready(function(){
     $("#asset_user_auth_view").modal("show");
 })
 .on('click', '.btn-update-auth', function() {
-    authUsername = $(this).data("user") ;
+    authUsername = $(this).data("username") ;
     authHostname = $(this).data("hostname");
     authAssetId = $(this).data("asset");
     $("#asset_user_auth_update_modal").modal('show');
 })
 .on("click", '.btn-test-auth', function () {
-    authUsername = $(this).data("user") ;
-    authAssetId = $(this).data("asset");
-    var the_url = "{% url 'api-assets:asset-user-connective' %}" + "?asset_id=" + authAssetId + "&username=" + authUsername;
-    if (prefer) {
-        the_url = setUrlParam(the_url, "prefer", prefer)
-    }
+    authUid = $(this).data('uid');
+    var theUrl = "{% url 'api-assets:asset-user-task-create'%}?id={{ DEFAULT_PK }}"
+        .replace("{{ DEFAULT_PK }}", authUid);
+
     var success = function (data) {
-        var task_id = data.task;
-        showCeleryTaskLog(task_id);
+        var taskId = data.task;
+        showCeleryTaskLog(taskId);
     };
     requestApi({
-        url: the_url,
-        method: 'GET',
+        url: theUrl,
+        method: 'POST',
+        data: JSON.stringify({action: 'test'}),
         success: success,
         flash_message: false
     });
 })
+.on('click', '.btn-del-auth', function () {
+    var uid = $(this).data("uid");
+    var theUrl = defaultAssetUserDetail.replace("{{ DEFAULT_PK }}", uid);
+    requestApi({
+        url: theUrl,
+        method: "DELETE",
+        success: function () {
+            assetUserTable.ajax.reload(null, false);
+        },
+        success_message: "{% trans 'Delete success' %}"
+    })
+
+})
+.on("change", '#only_latest', function () {
+    var checked = $("#only_latest").is(":checked");
+    if (checked === onlyLatestChecked) {
+        return
+    }
+    var ajaxUrl = assetUserTable.ajax.url();
+    if (checked) {
+        ajaxUrl = setUrlParam(ajaxUrl, 'latest', 1)
+    } else {
+        ajaxUrl = setUrlParam(ajaxUrl, 'latest', 0)
+    }
+    onlyLatestChecked = !onlyLatestChecked;
+    assetUserTable.ajax.url(ajaxUrl);
+    assetUserTable.ajax.reload();
+})
 
 
 </script>

apps/assets/templates/assets/_node_tree.html

@@ -113,10 +113,11 @@ function initNodeTree(options) {
     $.get(treeUrl, function (data, status) {
         zTree = $.fn.zTree.init($("#nodeTree"), setting, data);
         rootNodeAddDom(zTree, function () {
-            const url = '{% url 'api-assets:refresh-nodes-cache' %}';
+            const url = '{% url 'api-assets:node-task-create' pk=DEFAULT_PK %}';
             requestApi({
                 url: url,
-                method: 'GET',
+                method: 'POST',
+                data: {action: "refresh_cache"},
                 flash_message: false,
                 success: function () {
                     initNodeTree(options);
@@ -173,20 +174,14 @@ function removeTreeNode() {
 	if (!current_node){
 	    return
     }
-	if (current_node.children && current_node.children.length > 0) {
-		toastr.error("{% trans 'Have child node, cancel' %}");
-	 } else if (current_node.meta.node.assets_amount !== 0) {
-        toastr.error("{% trans 'Have assets, cancel' %}");
-    } else {
-        var url = "{% url 'api-assets:node-detail' pk=DEFAULT_PK %}".replace("{{ DEFAULT_PK }}", current_node_id);
-		$.ajax({
-            url: url,
-            method: "DELETE",
-            success: function () {
-                zTree.removeNode(current_node);
-            }
-        });
-	}
+    var url = "{% url 'api-assets:node-detail' pk=DEFAULT_PK %}".replace("{{ DEFAULT_PK }}", current_node_id);
+    requestApi({
+        url: url,
+        method: "DELETE",
+        success: function () {
+            zTree.removeNode(current_node)
+        }
+    })
 }
 
 function editTreeNode() {

apps/assets/templates/assets/_system_user.html

@@ -34,6 +34,7 @@
                             {% bootstrap_field form.name layout="horizontal" %}
                             {% bootstrap_field form.login_mode layout="horizontal" %}
                             {% bootstrap_field form.username layout="horizontal" %}
+                            {% bootstrap_field form.username_same_with_user layout="horizontal" %}
                             {% bootstrap_field form.priority layout="horizontal" %}
                             {% bootstrap_field form.protocol layout="horizontal" %}
 
@@ -63,6 +64,7 @@
                                 {% bootstrap_field form.cmd_filters layout="horizontal" %}
                             </div>
                             <h3>{% trans 'Other' %}</h3>
+                            {% bootstrap_field form.sftp_root layout="horizontal" %}
                             {% bootstrap_field form.sudo layout="horizontal" %}
                             {% bootstrap_field form.shell layout="horizontal" %}
                             {% bootstrap_field form.comment layout="horizontal" %}
@@ -226,6 +228,10 @@ $(document).ready(function () {
     $('.select2').select2();
     authFieldsDisplay();
     fieldDisplay();
+    var checked = $("#id_username_same_with_user").prop('checked');
+    if (checked) {
+        $("#id_username").attr("disabled", true)
+    }
 })
 .on('change', auto_generate_key, function(){
     authFieldsDisplay();
@@ -246,7 +252,7 @@ $(document).ready(function () {
     var data = form.serializeObject();
 
     objectAttrsIsList(data, ['cmd_filters']);
-    objectAttrsIsBool(data, ["auto_generate_key", "auto_push"]);
+    objectAttrsIsBool(data, ["auto_generate_key", "auto_push", "username_same_with_user"]);
     data["private_key"] = $("#id_private_key").data('file');
 
     var props = {
@@ -261,6 +267,15 @@ $(document).ready(function () {
     readFile($(this)).on("onload", function (evt, data) {
          $(this).data("file", data)
     })
+}).on("change", '#id_username_same_with_user', function () {
+    var checked = $(this).prop('checked');
+    var usernameRef = $("#id_username");
+    if (checked) {
+        usernameRef.val('');
+        usernameRef.attr("disabled", true)
+    } else {
+        usernameRef.attr("disabled", false)
+    }
 })
 
 </script>

apps/assets/templates/assets/admin_user_assets.html

@@ -72,9 +72,9 @@
 <script>
 
 $(document).ready(function () {
-	assetUserListUrl = setUrlParam(assetUserListUrl, "admin_user_id", "{{ admin_user.id }}");
-    prefer = "admin_user";
-    initAssetUserTable();
+	var assetUserListUrl = setUrlParam(defaultAssetUserListUrl, "prefer_id", "{{ admin_user.id }}");
+    assetUserListUrl = setUrlParam(assetUserListUrl, "prefer", "admin_user");
+    initAssetUserTable({assetUserListUrl: assetUserListUrl});
 })
 .on('click', '.btn-test-connective', function () {
     var the_url = "{% url 'api-assets:admin-user-connective' pk=admin_user.id %}";

apps/assets/templates/assets/admin_user_list.html

@@ -2,7 +2,7 @@
 {% load i18n static %}
 {% block help_message %}
     {% trans 'Admin users are asset (charged server) on the root, or have NOPASSWD: ALL sudo permissions users, '%}
-    {% trans 'Jumpserver users of the system using the user to `push system user`,  `get assets hardware information`, etc. '%}
+    {% trans 'JumpServer users of the system using the user to `push system user`,  `get assets hardware information`, etc. '%}
 {% endblock %}
 {% block table_search %}
     {% include '_csv_import_export.html' %}

apps/assets/templates/assets/asset_asset_user_list.html

@@ -73,19 +73,22 @@
 {% block custom_foot_js %}
 <script>
 $(document).ready(function () {
-    assetUserListUrl = setUrlParam(assetUserListUrl, "asset_id", "{{ asset.id }}");
-    initAssetUserTable()
+    assetUserListUrl = setUrlParam(defaultAssetUserListUrl, "asset_id", "{{ asset.id }}");
+    initAssetUserTable({assetUserListUrl: assetUserListUrl})
 })
 
 .on('click', '#btn-bulk-test-connective', function () {
-    var the_url = "{% url 'api-assets:asset-user-connective' %}" + "?asset_id={{ asset.id }}";
+    var assetId = "{{ asset.id }}";
+    var theUrl = "{% url 'api-assets:asset-user-task-create' %}?asset_id={{ DEFAULT_PK }}&latest=1"
+        .replace("{{ DEFAULT_PK }}", assetId);
     var success = function (data) {
         var task_id = data.task;
         showCeleryTaskLog(task_id);
     };
     requestApi({
-        url: the_url,
-        method: 'GET',
+        url: theUrl,
+        method: 'POST',
+        data: {action: "test"},
         success: success,
         flash_message: false
     });

apps/assets/templates/assets/asset_detail.html

@@ -268,16 +268,16 @@ function updateAssetNodes(nodes) {
 }
 
 function refreshAssetHardware() {
-    var the_url = "{% url 'api-assets:asset-refresh' pk=asset.id %}";
+    var the_url = "{% url 'api-assets:asset-task-create' pk=asset.id %}";
     var success = function(data) {
-        console.log(data);
         var task_id = data.task;
         showCeleryTaskLog(task_id);
     };
     requestApi({
         url: the_url,
         success: success,
-        method: 'GET'
+        data: {action: "refresh"},
+        method: 'POST'
     });
 }
 
@@ -345,16 +345,15 @@ $(document).ready(function () {
 }).on('click', '#btn_refresh_asset', function () {
     refreshAssetHardware()
 }).on('click', '#btn-test-is-alive', function () {
-    var the_url = "{% url 'api-assets:asset-alive-test' pk=asset.id %}";
-
+    var the_url = "{% url 'api-assets:asset-task-create' pk=asset.id %}";
     var success = function(data) {
         var task_id = data.task;
         showCeleryTaskLog(task_id);
     };
-
     requestApi({
         url: the_url,
-        method: 'GET',
+        method: 'POST',
+        data: {action: "test"},
         success: success
     });
 })

apps/assets/templates/assets/asset_list.html

@@ -350,7 +350,7 @@ $(document).ready(function(){
 }).on('click', '#menu_asset_move', function () {
     update_node_action = "move"
 }).on('click', '.btn-test-connective', function () {
-    var url = "{% url 'api-assets:node-test-connective' pk=DEFAULT_PK %}";
+    var url = "{% url 'api-assets:node-task-create' pk=DEFAULT_PK %}";
     if (!current_node_id) {
         return null;
     }
@@ -362,12 +362,13 @@ $(document).ready(function(){
     }
     requestApi({
         url: the_url,
-        method: "GET",
+        method: "POST",
+        data: {action: "test"},
         success: success,
         flash_message: false
     });
 }).on('click', '.btn-refresh-hardware', function () {
-    var url = "{% url 'api-assets:node-refresh-hardware-info' pk=DEFAULT_PK %}";
+    var url = "{% url 'api-assets:node-task-create' pk=DEFAULT_PK %}";
     var the_url = url.replace("{{ DEFAULT_PK }}", current_node_id);
     function success(data) {
         rMenu.css({"visibility" : "hidden"});
@@ -376,7 +377,8 @@ $(document).ready(function(){
     }
     requestApi({
         url: the_url,
-        method: "GET",
+        method: "POST",
+        data: {action: "refresh"},
         success: success,
         flash_message: false
     });

apps/assets/templates/assets/platform_detail.html

@@ -13,7 +13,12 @@
                                 <a href="{% url 'assets:platform-detail' pk=object.id %}" class="text-center"><i class="fa fa-laptop"></i> {% trans 'Detail' %} </a>
                             </li>
                             <li class="pull-right">
-                                <a class="btn btn-outline btn-default" href="{% url 'assets:platform-update' pk=object.id %}"><i class="fa fa-edit"></i>{% trans 'Update' %}</a>
+                                <a class="btn btn-outline btn-default" {% if object.internal %} disabled="true" {% endif %} href="{% url 'assets:platform-update' pk=object.id %}"><i class="fa fa-edit"></i>{% trans 'Update' %}</a>
+                            </li>
+                            <li class="pull-right">
+                                <a class="btn btn-outline btn-danger btn-del" {% if object.internal %} disabled="true" {% endif %}>
+                                    <i class="fa fa-trash-o"></i>{% trans 'Delete' %}
+                                </a>
                             </li>
                         </ul>
                     </div>
@@ -72,4 +77,15 @@
 {% endblock %}
 {% block content_bottom_left %}{% endblock %}
 {% block custom_foot_js %}
+<script>
+$(document).ready(function () {
+}).on('click', '.btn-del', function () {
+    var $this = $(this);
+    var name = "{{ object.name}}";
+    var uid = "{{ object.id }}";
+    var the_url = '{% url "api-assets:asset-platform-detail" pk=DEFAULT_PK %}'.replace('{{ DEFAULT_PK }}', uid);
+    var redirect_url = "{% url 'assets:platform-list' %}";
+    objectDelete($this, name, the_url, redirect_url);
+})
+</script>
 {% endblock %}

apps/assets/templates/assets/system_user_assets.html

@@ -23,16 +23,23 @@
                             </li>
                             <li class="active">
                                 <a href="{% url 'assets:system-user-asset' pk=system_user.id %}" class="text-center">
-                                <i class="fa fa-bar-chart-o"></i> {% trans 'Assets' %}
+                                <i class="fa fa-bar-chart-o"></i> {% trans 'Asset list' %}
                                 </a>
                             </li>
+                            {% if system_user.username_same_with_user  %}
+                            <li>
+                                <a href="{% url 'assets:system-user-user' pk=system_user.id %}" class="text-center">
+                                <i class="fa fa-bar-chart-o"></i> {% trans 'User list' %}
+                                </a>
+                            </li>
+                            {% endif %}
                         </ul>
                     </div>
                     <div class="tab-content">
                         <div class="col-sm-8" style="padding-left: 0;">
                             <div class="ibox float-e-margins">
                                 <div class="ibox-title">
-                                    <span style="float: left">{% trans 'Assets of ' %} <b>{{ system_user.name }} </b><span class="badge">{{ paginator.count }}</span></span>
+                                    <span style="float: left"><b>{{ system_user.name }} </b><span class="badge">{{ paginator.count }}</span></span>
                                     <div class="ibox-tools">
                                         <a class="collapse-link">
                                             <i class="fa fa-chevron-up"></i>
@@ -82,6 +89,30 @@
                                     </table>
                                 </div>
                             </div>
+                            <div class="panel panel-info">
+                                <div class="panel-heading">
+                                    <i class="fa fa-info-circle"></i> {% trans 'Assets' %}
+                                </div>
+                                <div class="panel-body">
+                                    <table class="table" id="add-assets">
+                                        <tbody>
+                                        <form>
+                                            <tr>
+                                                <td colspan="2" class="no-borders">
+                                                    <select data-placeholder="{% trans 'Select assets' %}" id="id_assets" class="assets-select2 select2" style="width: 100%" multiple="" tabindex="4">
+                                                    </select>
+                                                </td>
+                                            </tr>
+                                            <tr>
+                                                <td colspan="2" class="no-borders">
+                                                    <button type="button" class="btn btn-info btn-sm" id="btn-add-to-assets">{% trans 'Confirm' %}</button>
+                                                </td>
+                                            </tr>
+                                        </form>
+                                        </tbody>
+                                    </table>
+                                </div>
+                            </div>
                             <div class="panel panel-info">
                                 <div class="panel-heading">
                                     <i class="fa fa-info-circle"></i> {% trans 'Nodes' %}
@@ -92,7 +123,7 @@
                                         <form>
                                             <tr>
                                                 <td colspan="2" class="no-borders">
-                                                    <select data-placeholder="{% trans 'Add to node' %}" id="node_selected" class="nodes-select2" style="width: 100%" multiple="" tabindex="4">
+                                                    <select data-placeholder="{% trans 'Select nodes' %}" id="node_selected" class="nodes-select2" style="width: 100%" multiple="" tabindex="4">
                                                     </select>
                                                 </td>
                                             </tr>
@@ -114,6 +145,7 @@
             </div>
         </div>
     </div>
+    {% include 'assets/_asset_list_modal.html' %}
 {% endblock %}
 {% block custom_foot_js %}
 <script>
@@ -133,7 +165,7 @@ function getRelationUrl(type) {
     return theUrl;
 }
 
-function addObjects(objectsId, type) {
+function addObjects(objectsId, type, success, fail) {
     if (!objectsId || objectsId.length === 0) {
         return
     }
@@ -144,14 +176,22 @@ function addObjects(objectsId, type) {
         data[type] = v;
         body.push(data)
     });
-    requestApi({
+    if (!success) {
+        success = reloadPage
+    }
+    var option = {
         url: theUrl,
         body: JSON.stringify(body),
         method: "POST",
-        success: reloadPage
-    });
+        success: success,
+    };
+    if (fail) {
+        option.error = fail;
+    }
+    requestApi(option)
 }
 
+
 function removeObject(objectId, type, success) {
     if (!objectId)  {
         return
@@ -197,9 +237,13 @@ function initNodeTable() {
 $(document).ready(function () {
 	$('.select2').select2();
     nodesSelect2Init(".nodes-select2");
-	assetUserListUrl = setUrlParam(assetUserListUrl, "system_user_id", "{{ system_user.id }}");
-    needPush = true;
-    initAssetUserTable();
+    initAssetTreeModel('#id_assets');
+	var assetUserListUrl = setUrlParam(defaultAssetUserListUrl, "prefer_id", "{{ system_user.id }}");
+	assetUserListUrl = setUrlParam(assetUserListUrl, "prefer", "system_user");
+    initAssetUserTable({
+        assetUserListUrl: assetUserListUrl,
+        needPush: true
+    });
     initNodeTable();
 })
 .on('click', '.btn-remove-from-node', function() {
@@ -215,8 +259,31 @@ $(document).ready(function () {
     var nodes = $("#node_selected").val();
     addObjects(nodes, "node");
 })
+.on('click', '#btn-add-to-assets', function () {
+    var assets = $("#id_assets").val();
+    var options = $("#id_assets").find(":selected");
+    var failed = function(s, data) {
+        var invalidIndex = [];
+        var validIndex = [];
+        $.each(data, function (k, v) {
+            if (isEmptyObject(v)) {
+                validIndex.push(k)
+            } else {
+                invalidIndex.push(k)
+            }
+        });
+        var invalidLabel = [];
+        $.each(invalidIndex, function (k, v) {
+            invalidLabel.push(options[v].text)
+        });
+        var errorMsg = "{% trans 'Have existed: ' %}";
+        errorMsg += invalidLabel.join(", ");
+        toastr.error(errorMsg)
+    };
+    addObjects(assets, "asset", null, failed);
+})
 .on('click', '.btn-push', function () {
-    var theUrl = "{% url 'api-assets:system-user-push' pk=system_user.id %}";
+    var theUrl = "{% url 'api-assets:system-user-task-create' pk=system_user.id %}";
     var error = function (data) {
         alert(data)
     };
@@ -224,18 +291,26 @@ $(document).ready(function () {
         var taskId = data.task;
         showCeleryTaskLog(taskId);
     };
+    var data = {
+        action: 'push'
+    };
     requestApi({
         url: theUrl,
         error: error,
-        method: 'GET',
+        data: data,
+        method: 'POST',
         success: success
     });
 })
 .on('click', '.btn-push-auth', function () {
     var $this = $(this);
-    var asset_id = $this.data('asset');
-    var theUrl = "{% url 'api-assets:system-user-push-to-asset' pk=object.id aid=DEFAULT_PK %}";
-    theUrl = theUrl.replace("{{ DEFAULT_PK }}", asset_id);
+    var assetId = $this.data('asset');
+    var username = $this.data("username");
+    var theUrl = "{% url 'api-assets:system-user-task-create' pk=object.id %}?username=" + username;
+    var data = {
+        action: 'push',
+        asset: assetId,
+    };
     var success = function (data) {
         var taskId = data.task;
         showCeleryTaskLog(taskId);
@@ -245,13 +320,15 @@ $(document).ready(function () {
     };
     requestApi({
         url: theUrl,
-        method: 'GET',
+        method: 'POST',
+        data: data,
         success: success,
+        flash_message: false,
         error: error
     })
 })
 .on('click', '.btn-test-connective', function () {
-    var theUrl = "{% url 'api-assets:system-user-connective' pk=system_user.id %}";
+    var theUrl = "{% url 'api-assets:system-user-task-create' pk=system_user.id %}";
     var error = function (data) {
         alert(data)
     };
@@ -261,9 +338,11 @@ $(document).ready(function () {
     };
     requestApi({
         url: theUrl,
+        data: {action: "test"},
         error: error,
-        method: 'GET',
-        success: success
+        method: 'POST',
+        success: success,
+        flash_message: false,
     });
 })
 </script>

apps/assets/templates/assets/system_user_detail.html

@@ -15,7 +15,14 @@
                             {% if system_user.can_perm_to_asset  %}
                             <li>
                                 <a href="{% url 'assets:system-user-asset' pk=system_user.id %}" class="text-center">
-                                <i class="fa fa-bar-chart-o"></i> {% trans 'Assets' %}
+                                <i class="fa fa-bar-chart-o"></i> {% trans 'Asset list' %}
+                                </a>
+                            </li>
+                            {% endif %}
+                            {% if system_user.username_same_with_user  %}
+                            <li>
+                                <a href="{% url 'assets:system-user-user' pk=system_user.id %}" class="text-center">
+                                <i class="fa fa-bar-chart-o"></i> {% trans 'User list' %}
                                 </a>
                             </li>
                             {% endif %}
@@ -57,7 +64,11 @@
                                         </tr>
                                         <tr>
                                             <td>{% trans 'Username' %}:</td>
-                                            <td><b>{{ system_user.username }}</b></td>
+                                            {% if system_user.username_same_with_user %}
+                                                <td><b>{% trans 'Username same with user' %}</b></td>
+                                            {% else %}
+                                                <td><b>{{ system_user.username }}</b></td>
+                                            {% endif %}
                                         </tr>
                                         <tr>
                                             <td>{% trans 'Login mode' %}:</td>
@@ -131,26 +142,6 @@
                                                 </span>
                                             </td>
                                         </tr>
-                                        {% if system_user.auto_push %}
-                                        <tr class="only-ssh-rdp">
-                                            <td width="50%">{% trans 'Push system user now' %}:</td>
-                                            <td>
-                                                <span style="float: right">
-                                                    <button type="button" class="btn btn-primary btn-xs btn-push" style="width: 54px">{% trans 'Push' %}</button>
-                                                </span>
-                                            </td>
-                                        </tr>
-                                        {% endif %}
-                                        {% if system_user.is_need_test_asset_connective %}
-                                        <tr>
-                                            <td width="50%">{% trans 'Test assets connective' %}:</td>
-                                            <td>
-                                                <span style="float: right">
-                                                    <button type="button" class="btn btn-primary btn-xs btn-test-connective" style="width: 54px">{% trans 'Test' %}</button>
-                                                </span>
-                                            </td>
-                                        </tr>
-                                        {% endif %}
                                         </tbody>
                                     </table>
                                 </div>
@@ -246,32 +237,7 @@ $(document).ready(function () {
     var redirect_url = "{% url 'assets:system-user-list' %}";
     objectDelete($this, name, the_url, redirect_url);
 })
-.on('click', '.btn-push', function () {
-    var the_url = "{% url 'api-assets:system-user-push' pk=system_user.id %}";
-    var success = function (data) {
-        var task_id = data.task;
-        showCeleryTaskLog(task_id);
-    };
-    requestApi({
-        url: the_url,
-        method: 'GET',
-        success: success,
-        flash_message: false
-    });
-})
-.on('click', '.btn-test-connective', function () {
-    var the_url = "{% url 'api-assets:system-user-connective' pk=system_user.id %}";
-    var success = function (data) {
-        var task_id = data.task;
-        showCeleryTaskLog(task_id);
-    };
-    requestApi({
-        url: the_url,
-        method: 'GET',
-        success: success,
-        flash_message: false
-    });
-}).on('click', '#btn-binding-command-filters', function () {
+.on('click', '#btn-binding-command-filters', function () {
     var new_selected_cmd_filters = $.map($('#command_filters_selected').select2('data'), function (i) {
         return i.id;
     });

apps/assets/templates/assets/system_user_list.html

@@ -2,9 +2,9 @@
 {% load i18n %}
 
 {% block help_message %}
-    {% trans 'System user is Jumpserver jump login assets used by the users, can be understood as the user login assets, such as web, sa, the dba (` ssh web@some-host `), rather than using a user the username login server jump (` ssh xiaoming@some-host `); '%}
-    {% trans 'In simple terms, users log into Jumpserver using their own username, and Jumpserver uses system users to log into assets. '%}
-    {% trans 'When system users are created, if you choose auto push Jumpserver to use Ansible push system users into the asset, if the asset (Switch) does not support ansible, please manually fill in the account password.' %}
+    {% trans 'System user is JumpServer jump login assets used by the users, can be understood as the user login assets, such as web, sa, the dba (` ssh web@some-host `), rather than using a user the username login server jump (` ssh xiaoming@some-host `); '%}
+    {% trans 'In simple terms, users log into JumpServer using their own username, and JumpServer uses system users to log into assets. '%}
+    {% trans 'When system users are created, if you choose auto push JumpServer to use Ansible push system users into the asset, if the asset (Switch) does not support ansible, please manually fill in the account password.' %}
 {% endblock %}
 
 {% block table_search %}
@@ -26,9 +26,6 @@
         <th class="text-center">{% trans 'Protocol' %}</th>
         <th class="text-center">{% trans 'Login mode' %}</th>
         <th class="text-center">{% trans 'Asset' %}</th>
-{#        <th class="text-center">{% trans 'Reachable' %}</th>#}
-{#        <th class="text-center">{% trans 'Unreachable' %}</th>#}
-{#        <th class="text-center">{% trans 'Ratio' %}</th>#}
         <th class="text-center">{% trans 'Comment' %}</th>
         <th class="text-center">{% trans 'Action' %}</th>
     </tr>
@@ -40,6 +37,8 @@
 {% block custom_foot_js %}
 <script>
 var system_user_table = 0;
+
+
 function initTable() {
     var options = {
         ele: $('#system_user_list_table'),
@@ -61,7 +60,7 @@ function initTable() {
         ajax_url: '{% url "api-assets:system-user-list" %}',
         columns: [
             {data: "id" }, {data: "name" }, {data: "username" }, {data: "protocol"},
-            {data: "login_mode"}, {data: "assets_amount", orderable: false },
+            {data: "login_mode"}, {data: "assets_amount", width: "60px"},
             {data: "comment" }, {data: "id", orderable: false, width: "120px"}
         ],
         op_html: $('#actions').html()

apps/assets/templates/assets/system_user_users.html

@@ -0,0 +1,212 @@
+{% extends 'base.html' %}
+{% load static %}
+{% load i18n %}
+
+{% block custom_head_css_js %}
+    <style>
+    .table.node_edit {
+        margin-bottom: 0;
+    }
+    </style>
+{% endblock %}
+
+{% block content %}
+    <div class="wrapper wrapper-content animated fadeInRight">
+        <div class="row">
+            <div class="col-sm-12">
+                <div class="ibox float-e-margins">
+                    <div class="panel-options">
+                        <ul class="nav nav-tabs">
+                            <li>
+                                <a href="{% url 'assets:system-user-detail' pk=system_user.id %}" class="text-center"><i class="fa fa-laptop"></i> {% trans 'Detail' %} </a>
+                            </li>
+                            <li>
+                                <a href="{% url 'assets:system-user-asset' pk=system_user.id %}" class="text-center">
+                                <i class="fa fa-bar-chart-o"></i> {% trans 'Asset list' %}
+                                </a>
+                            </li>
+                            {% if system_user.username_same_with_user  %}
+                            <li class="active">
+                                <a href="{% url 'assets:system-user-user' pk=system_user.id %}" class="text-center">
+                                <i class="fa fa-bar-chart-o"></i> {% trans 'User list' %}
+                                </a>
+                            </li>
+                            {% endif %}
+                        </ul>
+                    </div>
+                    <div class="tab-content">
+                        <div class="col-sm-8" style="padding-left: 0;">
+                            <div class="ibox float-e-margins">
+                                <div class="ibox-title">
+                                    <span style="float: left"><b>{{ system_user.name }} </b><span class="badge">{{ paginator.count }}</span></span>
+                                    <div class="ibox-tools">
+                                        <a class="collapse-link">
+                                            <i class="fa fa-chevron-up"></i>
+                                        </a>
+                                        <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+                                            <i class="fa fa-wrench"></i>
+                                        </a>
+                                        <ul class="dropdown-menu dropdown-user">
+                                        </ul>
+                                        <a class="close-link">
+                                            <i class="fa fa-times"></i>
+                                        </a>
+                                    </div>
+                                </div>
+                                <div class="ibox-content">
+                                    <table class="table table-striped table-bordered table-hover" id="user_list_table" style="width: 100%">
+                                        <thead>
+                                        <tr>
+                                            <th class="text-center">
+                                                <input type="checkbox" id="check_all" class="ipt_check_all">
+                                            </th>
+                                            <th class="text-center">{% trans 'User' %}</th>
+                                            <th class="text-center">{% trans 'Action' %}</th>
+                                        </tr>
+                                        </thead>
+                                        <tbody>
+                                        </tbody>
+                                    </table>
+                                </div>
+                            </div>
+                        </div>
+                        <div class="col-sm-4" style="padding-left: 0;padding-right: 0">
+                            <div class="panel panel-info">
+                                <div class="panel-heading">
+                                    <i class="fa fa-info-circle"></i> {% trans 'Users' %}
+                                </div>
+                                <div class="panel-body">
+                                    <table class="table" id="add-users">
+                                        <tbody>
+                                        <form>
+                                            <tr>
+                                                <td colspan="2" class="no-borders">
+                                                    <select data-placeholder="{% trans 'Select users' %}" id="id_users" class="users-select2 select2" style="width: 100%" multiple="" tabindex="4">
+                                                    </select>
+                                                </td>
+                                            </tr>
+                                            <tr>
+                                                <td colspan="2" class="no-borders">
+                                                    <button type="button" class="btn btn-info btn-sm" id="btn-add-users">{% trans 'Confirm' %}</button>
+                                                </td>
+                                            </tr>
+                                        </form>
+                                        </tbody>
+                                    </table>
+                                </div>
+                            </div>
+                        </div>
+                    </div>
+                </div>
+            </div>
+        </div>
+    </div>
+{% endblock %}
+{% block custom_foot_js %}
+<script>
+var usersRelationUrl = "{% url 'api-assets:system-users-users-relation-list' %}?systemuser={{ system_user.id }}";
+var userTable = null;
+
+function initTable() {
+    var options = {
+        ele: $('#user_list_table'),
+        toggle: true,
+        ajax_url: usersRelationUrl,
+        columnDefs: [
+            {
+                targets: 2, createdCell: function (td, cellData, rowData) {
+                    var removeBtn = '<button class="btn btn-xs btn-danger m-l-xs btn-remove" data-uid=UID>{% trans "Remove" %}</button>';
+                    removeBtn = removeBtn.replace("UID", rowData.user);
+				    $(td).html(removeBtn);
+                },
+                width: '70px'
+            }
+        ],
+        columns: [
+            {data: "id"}, {data: "user_display"},
+            {data: "id", orderable: false}
+        ],
+        op_html: $('#actions').html(),
+    };
+    userTable = jumpserver.initServerSideDataTable(options);
+    return userTable
+}
+
+function addUsers(objectsId, success, fail) {
+    if (!objectsId || objectsId.length === 0) {
+        return
+    }
+    var theUrl = usersRelationUrl;
+    var body = [];
+    objectsId.forEach(function (v) {
+        var data = {systemuser: "{{ object.id }}"};
+        data.user = v;
+        body.push(data)
+    });
+    if (!success) {
+        success = reloadPage
+    }
+    var option = {
+        url: theUrl,
+        body: JSON.stringify(body),
+        method: "POST",
+        success: success,
+    };
+    if (fail) {
+        option.error = fail;
+    }
+    requestApi(option)
+}
+
+function removeUser(userId, success) {
+    if (!userId)  {
+        return
+    }
+    var theUrl = usersRelationUrl;
+    theUrl = setUrlParam(theUrl, 'user', userId);
+    if (!success) {
+        success = function () {
+            userTable.ajax.reload();
+        }
+    }
+    requestApi({
+        url: theUrl,
+        method: "DELETE",
+        success: success,
+        success_message: "{% trans "Remove success" %}"
+    });
+}
+
+$(document).ready(function () {
+	initTable();
+	usersSelect2Init('.users-select2');
+})
+.on('click', '.btn-remove', function() {
+    var userId = $(this).data("uid");
+    removeUser(userId);
+})
+.on('click', '#btn-add-users', function() {
+    var usersId = $('.users-select2').val();
+    var options = $(".users-select2").find(":selected");
+    var failed = function(s, data) {
+        var invalidIndex = [];
+        var validIndex = [];
+        $.each(data, function (k, v) {
+            if (isEmptyObject(v)) {
+                validIndex.push(k)
+            } else {
+                invalidIndex.push(k)
+            }
+        });
+        var invalidLabel = [];
+        $.each(invalidIndex, function (k, v) {
+            invalidLabel.push(options[v].text)
+        });
+        var errorMsg = "{% trans 'Have existed: ' %}";
+        errorMsg += invalidLabel.join(", ");
+        toastr.error(errorMsg)
+    };
+    addUsers(usersId, null, failed);
+})
+</script>
+{% endblock %}

apps/assets/tests.py

@@ -1,3 +0,0 @@
-from django.test import TestCase
-
-# Create your tests here.

apps/assets/tests/__init__.py

@@ -0,0 +1,2 @@
+# -*- coding: utf-8 -*-
+#

apps/assets/tests/test_system_user.py

@@ -0,0 +1,2 @@
+# -*- coding: utf-8 -*-
+#

apps/assets/urls/api_urls.py

@@ -21,82 +21,46 @@ router.register(r'domains', api.DomainViewSet, 'domain')
 router.register(r'gateways', api.GatewayViewSet, 'gateway')
 router.register(r'cmd-filters', api.CommandFilterViewSet, 'cmd-filter')
 router.register(r'asset-users', api.AssetUserViewSet, 'asset-user')
-router.register(r'asset-users-info', api.AssetUserExportViewSet, 'asset-user-info')
+router.register(r'asset-user-auth-infos', api.AssetUserAuthInfoViewSet, 'asset-user-auth-info')
 router.register(r'gathered-users', api.GatheredUserViewSet, 'gathered-user')
 router.register(r'favorite-assets', api.FavoriteAssetViewSet, 'favorite-asset')
 router.register(r'system-users-assets-relations', api.SystemUserAssetRelationViewSet, 'system-users-assets-relation')
 router.register(r'system-users-nodes-relations', api.SystemUserNodeRelationViewSet, 'system-users-nodes-relation')
+router.register(r'system-users-users-relations', api.SystemUserUserRelationViewSet, 'system-users-users-relation')
 
 cmd_filter_router = routers.NestedDefaultRouter(router, r'cmd-filters', lookup='filter')
 cmd_filter_router.register(r'rules', api.CommandFilterRuleViewSet, 'cmd-filter-rule')
 
 
 urlpatterns = [
-    path('assets/<uuid:pk>/refresh/',
-         api.AssetRefreshHardwareApi.as_view(), name='asset-refresh'),
-    path('assets/<uuid:pk>/alive/',
-         api.AssetAdminUserTestApi.as_view(), name='asset-alive-test'),
-    path('assets/<uuid:pk>/gateway/',
-         api.AssetGatewayApi.as_view(), name='asset-gateway'),
-    path('assets/<uuid:pk>/platform/',
-         api.AssetPlatformRetrieveApi.as_view(), name='asset-platform-detail'),
-
-    path('asset-users/auth-info/',
-         api.AssetUserAuthInfoApi.as_view(), name='asset-user-auth-info'),
-    path('asset-users/test-connective/',
-         api.AssetUserTestConnectiveApi.as_view(), name='asset-user-connective'),
-
-
-    path('admin-users/<uuid:pk>/nodes/',
-         api.ReplaceNodesAdminUserApi.as_view(), name='replace-nodes-admin-user'),
-    path('admin-users/<uuid:pk>/auth/',
-         api.AdminUserAuthApi.as_view(), name='admin-user-auth'),
-    path('admin-users/<uuid:pk>/connective/',
-         api.AdminUserTestConnectiveApi.as_view(), name='admin-user-connective'),
-    path('admin-users/<uuid:pk>/assets/',
-         api.AdminUserAssetsListView.as_view(), name='admin-user-assets'),
-
-    path('system-users/<uuid:pk>/auth-info/',
-         api.SystemUserAuthInfoApi.as_view(), name='system-user-auth-info'),
-    path('system-users/<uuid:pk>/assets/<uuid:aid>/auth-info/',
-         api.SystemUserAssetAuthInfoApi.as_view(), name='system-user-asset-auth-info'),
-    path('system-users/<uuid:pk>/assets/',
-         api.SystemUserAssetsListView.as_view(), name='system-user-assets'),
-    path('system-users/<uuid:pk>/push/',
-         api.SystemUserPushApi.as_view(), name='system-user-push'),
-    path('system-users/<uuid:pk>/assets/<uuid:aid>/push/',
-         api.SystemUserPushToAssetApi.as_view(), name='system-user-push-to-asset'),
-    path('system-users/<uuid:pk>/assets/<uuid:aid>/test/',
-         api.SystemUserTestAssetConnectivityApi.as_view(), name='system-user-test-to-asset'),
-    path('system-users/<uuid:pk>/connective/',
-         api.SystemUserTestConnectiveApi.as_view(), name='system-user-connective'),
-    path('system-users/<uuid:pk>/cmd-filter-rules/',
-         api.SystemUserCommandFilterRuleListApi.as_view(), name='system-user-cmd-filter-rule-list'),
+    path('assets/<uuid:pk>/gateways/', api.AssetGatewayListApi.as_view(), name='asset-gateway-list'),
+    path('assets/<uuid:pk>/platform/', api.AssetPlatformRetrieveApi.as_view(), name='asset-platform-detail'),
+    path('assets/<uuid:pk>/tasks/', api.AssetTaskCreateApi.as_view(), name='asset-task-create'),
+
+    path('asset-users/tasks/', api.AssetUserTaskCreateAPI.as_view(), name='asset-user-task-create'),
+
+    path('admin-users/<uuid:pk>/nodes/', api.ReplaceNodesAdminUserApi.as_view(), name='replace-nodes-admin-user'),
+    path('admin-users/<uuid:pk>/auth/', api.AdminUserAuthApi.as_view(), name='admin-user-auth'),
+    path('admin-users/<uuid:pk>/connective/', api.AdminUserTestConnectiveApi.as_view(), name='admin-user-connective'),
+    path('admin-users/<uuid:pk>/assets/', api.AdminUserAssetsListView.as_view(), name='admin-user-assets'),
+
+    path('system-users/<uuid:pk>/auth-info/', api.SystemUserAuthInfoApi.as_view(), name='system-user-auth-info'),
+    path('system-users/<uuid:pk>/assets/<uuid:aid>/auth-info/', api.SystemUserAssetAuthInfoApi.as_view(), name='system-user-asset-auth-info'),
+    path('system-users/<uuid:pk>/tasks/', api.SystemUserTaskApi.as_view(), name='system-user-task-create'),
+    path('system-users/<uuid:pk>/cmd-filter-rules/', api.SystemUserCommandFilterRuleListApi.as_view(), name='system-user-cmd-filter-rule-list'),
 
     path('nodes/tree/', api.NodeListAsTreeApi.as_view(), name='node-tree'),
     path('nodes/children/tree/', api.NodeChildrenAsTreeApi.as_view(), name='node-children-tree'),
-    path('nodes/<uuid:pk>/children/',
-         api.NodeChildrenApi.as_view(), name='node-children'),
+    path('nodes/<uuid:pk>/children/', api.NodeChildrenApi.as_view(), name='node-children'),
     path('nodes/children/', api.NodeChildrenApi.as_view(), name='node-children-2'),
-    path('nodes/<uuid:pk>/children/add/',
-         api.NodeAddChildrenApi.as_view(), name='node-add-children'),
-    path('nodes/<uuid:pk>/assets/',
-         api.NodeAssetsApi.as_view(), name='node-assets'),
-    path('nodes/<uuid:pk>/assets/add/',
-         api.NodeAddAssetsApi.as_view(), name='node-add-assets'),
-    path('nodes/<uuid:pk>/assets/replace/',
-         api.NodeReplaceAssetsApi.as_view(), name='node-replace-assets'),
-    path('nodes/<uuid:pk>/assets/remove/',
-         api.NodeRemoveAssetsApi.as_view(), name='node-remove-assets'),
-    path('nodes/<uuid:pk>/refresh-hardware-info/',
-         api.RefreshNodeHardwareInfoApi.as_view(), name='node-refresh-hardware-info'),
-    path('nodes/<uuid:pk>/test-connective/',
-         api.TestNodeConnectiveApi.as_view(), name='node-test-connective'),
-
-    path('nodes/cache/', api.RefreshNodesCacheApi.as_view(), name='refresh-nodes-cache'),
-
-    path('gateways/<uuid:pk>/test-connective/',
-         api.GatewayTestConnectionApi.as_view(), name='test-gateway-connective'),
+    path('nodes/<uuid:pk>/children/add/', api.NodeAddChildrenApi.as_view(), name='node-add-children'),
+    path('nodes/<uuid:pk>/assets/', api.NodeAssetsApi.as_view(), name='node-assets'),
+    path('nodes/<uuid:pk>/assets/add/', api.NodeAddAssetsApi.as_view(), name='node-add-assets'),
+    path('nodes/<uuid:pk>/assets/replace/', api.NodeReplaceAssetsApi.as_view(), name='node-replace-assets'),
+    path('nodes/<uuid:pk>/assets/remove/', api.NodeRemoveAssetsApi.as_view(), name='node-remove-assets'),
+    path('nodes/<uuid:pk>/tasks/', api.NodeTaskCreateApi.as_view(), name='node-task-create'),
+
+    path('gateways/<uuid:pk>/test-connective/', api.GatewayTestConnectionApi.as_view(), name='test-gateway-connective'),
 
 ]
 

apps/assets/urls/views_urls.py

@@ -39,6 +39,7 @@ urlpatterns = [
     path('system-user/<uuid:pk>/update/', views.SystemUserUpdateView.as_view(), name='system-user-update'),
     path('system-user/<uuid:pk>/delete/', views.SystemUserDeleteView.as_view(), name='system-user-delete'),
     path('system-user/<uuid:pk>/asset/', views.SystemUserAssetView.as_view(), name='system-user-asset'),
+    path('system-user/<uuid:pk>/user/', views.SystemUserUserView.as_view(), name='system-user-user'),
 
     path('label/', views.LabelListView.as_view(), name='label-list'),
     path('label/create/', views.LabelCreateView.as_view(), name='label-create'),

apps/assets/utils.py

@@ -5,68 +5,53 @@ from treelib.exceptions import NodeIDAbsentError
 from collections import defaultdict
 from copy import deepcopy
 
-from common.utils import get_object_or_none, get_logger, timeit
-from .models import SystemUser, Asset
+from common.utils import get_logger, timeit, lazyproperty
+from .models import Asset, Node
 
 
 logger = get_logger(__file__)
 
 
-def get_system_user_by_name(name):
-    system_user = get_object_or_none(SystemUser, name=name)
-    return system_user
-
-
-def get_system_user_by_id(id):
-    system_user = get_object_or_none(SystemUser, id=id)
-    return system_user
-
-
 class TreeService(Tree):
     tag_sep = ' / '
 
-    def __init__(self, *args, **kwargs):
-        super().__init__(*args, **kwargs)
-        self.nodes_assets_map = defaultdict(set)
-        self.all_nodes_assets_map = {}
-        self._invalid_assets = frozenset()
+    @staticmethod
+    @timeit
+    def get_nodes_assets_map():
+        nodes_assets_map = defaultdict(set)
+        asset_node_list = Node.assets.through.objects.values_list(
+            'asset', 'node__key'
+        )
+        for asset_id, key in asset_node_list:
+            nodes_assets_map[key].add(asset_id)
+        return nodes_assets_map
 
     @classmethod
     @timeit
     def new(cls):
         from .models import Node
-        from orgs.utils import tmp_to_root_org
-
-        with tmp_to_root_org():
-            all_nodes = list(Node.objects.all().values("key", "value"))
-            all_nodes.sort(key=lambda x: len(x["key"].split(":")))
-            tree = cls()
-            tree.create_node(tag='', identifier='')
-            for node in all_nodes:
-                key = node["key"]
-                value = node["value"]
-                parent_key = ":".join(key.split(":")[:-1])
-                tree.safe_create_node(
-                    tag=value, identifier=key,
-                    parent=parent_key,
-                )
-            tree.init_assets()
+        all_nodes = list(Node.objects.all().values("key", "value"))
+        all_nodes.sort(key=lambda x: len(x["key"].split(":")))
+        tree = cls()
+        tree.create_node(tag='', identifier='', data={})
+        for node in all_nodes:
+            key = node["key"]
+            value = node["value"]
+            parent_key = ":".join(key.split(":")[:-1])
+            tree.safe_create_node(
+                tag=value, identifier=key,
+                parent=parent_key,
+            )
+        tree.init_assets()
         return tree
 
-    @timeit
     def init_assets(self):
-        from orgs.utils import tmp_to_root_org
-        self.all_nodes_assets_map = {}
-        self.nodes_assets_map = defaultdict(set)
-        with tmp_to_root_org():
-            queryset = Asset.objects.all().values_list('id', 'nodes__key')
-            invalid_assets = Asset.objects.filter(is_active=False)\
-                .values_list('id', flat=True)
-            self._invalid_assets = frozenset(invalid_assets)
-            for asset_id, key in queryset:
-                if not key:
-                    continue
-                self.nodes_assets_map[key].add(asset_id)
+        node_assets_map = self.get_nodes_assets_map()
+        for node in self.all_nodes_itr():
+            key = node.identifier
+            assets = node_assets_map.get(key, set())
+            data = {"assets": assets, "all_assets": None}
+            node.data = data
 
     def safe_create_node(self, **kwargs):
         parent = kwargs.get("parent")
@@ -125,32 +110,43 @@ class TreeService(Tree):
             parent = self.copy_node(parent)
         return parent
 
+    @lazyproperty
+    def invalid_assets(self):
+        assets = Asset.objects.filter(is_active=False).values_list('id', flat=True)
+        return assets
+
     def set_assets(self, nid, assets):
-        self.nodes_assets_map[nid] = set(assets)
+        node = self.get_node(nid)
+        if node.data is None:
+            node.data = {}
+        node.data["assets"] = assets
 
     def assets(self, nid):
-        assets = self.nodes_assets_map[nid]
-        return assets
+        node = self.get_node(nid)
+        return node.data.get("assets", set())
 
     def valid_assets(self, nid):
-        return set(self.assets(nid)) - set(self._invalid_assets)
+        return set(self.assets(nid)) - set(self.invalid_assets)
 
     def all_assets(self, nid):
-        assets = self.all_nodes_assets_map.get(nid)
-        if assets:
-            return assets
-        assets = set(self.assets(nid))
+        node = self.get_node(nid)
+        if node.data is None:
+            node.data = {}
+        all_assets = node.data.get("all_assets")
+        if all_assets is not None:
+            return all_assets
+        all_assets = set(self.assets(nid))
         try:
             children = self.children(nid)
         except NodeIDAbsentError:
             children = []
         for child in children:
-            assets.update(self.all_assets(child.identifier))
-        self.all_nodes_assets_map[nid] = assets
-        return assets
+            all_assets.update(self.all_assets(child.identifier))
+        node.data["all_assets"] = all_assets
+        return all_assets
 
     def all_valid_assets(self, nid):
-        return set(self.all_assets(nid)) - set(self._invalid_assets)
+        return set(self.all_assets(nid)) - set(self.invalid_assets)
 
     def assets_amount(self, nid):
         return len(self.all_assets(nid))
@@ -186,15 +182,12 @@ class TreeService(Tree):
         else:
             # logger.debug('Add node: {}'.format(node.identifier))
             self.add_node(node, parent)
-
     #
     # def __getstate__(self):
     #     self.mutex = None
+    #     self.all_nodes_assets_map = {}
+    #     self.nodes_assets_map = {}
     #     return self.__dict__
-    #
 
-    def __setstate__(self, state):
-        self.__dict__ = state
-        if '_invalid_assets' not in state:
-            self._invalid_assets = frozenset()
-        # self.mutex = threading.Lock()
+    # def __setstate__(self, state):
+    #     self.__dict__ = state

apps/assets/views/admin_user.py

@@ -106,7 +106,7 @@ class AdminUserAssetsView(PermissionsMixin, SingleObjectMixin, ListView):
     def get_context_data(self, **kwargs):
         context = {
             'app': _('Assets'),
-            'action': _('Admin user detail'),
+            'action': _('Admin user assets'),
         }
         kwargs.update(context)
         return super().get_context_data(**kwargs)

apps/assets/views/platform.py

@@ -48,6 +48,9 @@ class PlatformUpdateView(generic.UpdateView):
     model = Platform
     template_name = 'assets/platform_create_update.html'
 
+    def post(self, *args, **kwargs):
+        pass
+
     def get_context_data(self, **kwargs):
         context = super().get_context_data(**kwargs)
         meta_form = PlatformMetaForm(initial=self.object.meta)

apps/assets/views/system_user.py

@@ -17,6 +17,7 @@ __all__ = [
     'SystemUserCreateView', 'SystemUserUpdateView',
     'SystemUserDetailView', 'SystemUserDeleteView',
     'SystemUserAssetView', 'SystemUserListView',
+    'SystemUserUserView',
 ]
 
 
@@ -100,7 +101,22 @@ class SystemUserAssetView(PermissionsMixin, DetailView):
     def get_context_data(self, **kwargs):
         context = {
             'app': _('assets'),
-            'action': _('System user asset'),
+            'action': _('System user assets'),
+        }
+        kwargs.update(context)
+        return super().get_context_data(**kwargs)
+
+
+class SystemUserUserView(PermissionsMixin, DetailView):
+    model = SystemUser
+    template_name = 'assets/system_user_users.html'
+    context_object_name = 'system_user'
+    permission_classes = [IsOrgAdmin]
+
+    def get_context_data(self, **kwargs):
+        context = {
+            'app': _('assets'),
+            'action': _('System user users'),
         }
         kwargs.update(context)
         return super().get_context_data(**kwargs)

apps/authentication/backends/cas/__init__.py

@@ -0,0 +1,4 @@
+# -*- coding: utf-8 -*-
+#
+from .backends import *
+from .callback import *

apps/authentication/backends/cas/backends.py

@@ -0,0 +1,11 @@
+# -*- coding: utf-8 -*-
+#
+from django_cas_ng.backends import CASBackend as _CASBackend
+
+
+__all__ = ['CASBackend']
+
+
+class CASBackend(_CASBackend):
+    def user_can_authenticate(self, user):
+        return True

apps/authentication/backends/cas/callback.py

@@ -0,0 +1,16 @@
+# -*- coding: utf-8 -*-
+#
+from django.contrib.auth import get_user_model
+
+
+User = get_user_model()
+
+
+def cas_callback(response):
+    username = response['username']
+    user, user_created = User.objects.get_or_create(username=username)
+    profile, created = user.get_profile()
+
+    profile.role = response['attributes']['role']
+    profile.birth_date = response['attributes']['birth_date']
+    profile.save()

apps/authentication/backends/cas/urls.py

@@ -0,0 +1,11 @@
+# -*- coding: utf-8 -*-
+#
+from django.urls import path
+import django_cas_ng.views
+
+
+urlpatterns = [
+    path('login/', django_cas_ng.views.LoginView.as_view(), name='cas-login'),
+    path('logout/', django_cas_ng.views.LogoutView.as_view(), name='cas-logout'),
+    path('callback/', django_cas_ng.views.CallbackView.as_view(), name='cas-proxy-callback'),
+]

apps/authentication/backends/ldap.py

@@ -29,26 +29,27 @@ class LDAPAuthorizationBackend(LDAPBackend):
 
     def pre_check(self, username, password):
         if not settings.AUTH_LDAP:
-            return False
-        logger.info('Authentication LDAP backend')
+            error = 'Not enabled auth ldap'
+            return False, error
         if not username:
-            logger.info('Authenticate failed: username is None')
-            return False
+            error = 'Username is None'
+            return False, error
         if not password:
-            logger.info('Authenticate failed: password is None')
-            return False
+            error = 'Password is None'
+            return False, error
         if settings.AUTH_LDAP_USER_LOGIN_ONLY_IN_USERS:
             user_model = self.get_user_model()
             exist = user_model.objects.filter(username=username).exists()
             if not exist:
-                msg = 'Authentication failed: user ({}) is not in the user list'
-                logger.info(msg.format(username))
-                return False
-        return True
+                error = 'user ({}) is not in the user list'.format(username)
+                return False, error
+        return True, ''
 
     def authenticate(self, request=None, username=None, password=None, **kwargs):
-        match = self.pre_check(username, password)
+        logger.info('Authentication LDAP backend')
+        match, msg = self.pre_check(username, password)
         if not match:
+            logger.info('Authenticate failed: {}'.format(msg))
             return None
         ldap_user = LDAPUser(self, username=username.strip(), request=request)
         user = self.authenticate_ldap_user(ldap_user, password)
@@ -130,5 +131,5 @@ class LDAPUser(_LDAPUser):
                 setattr(self._user, field, value)
 
         email = getattr(self._user, 'email', '')
-        email = construct_user_email(email, self._user.username)
+        email = construct_user_email(self._user.username, email)
         setattr(self._user, 'email', email)

apps/authentication/backends/pubkey.py

@@ -19,7 +19,7 @@ class PublicKeyAuthBackend:
             return None
         else:
             if user.check_public_key(public_key) and \
-                    self.user_can_authenticate(user):
+                  self.user_can_authenticate(user):
                 return user
 
     @staticmethod

apps/authentication/errors.py

@@ -11,6 +11,7 @@ from users.utils import (
 
 reason_password_failed = 'password_failed'
 reason_mfa_failed = 'mfa_failed'
+reason_mfa_unset = 'mfa_unset'
 reason_user_not_exist = 'user_not_exist'
 reason_password_expired = 'password_expired'
 reason_user_invalid = 'user_invalid'
@@ -18,7 +19,8 @@ reason_user_inactive = 'user_inactive'
 
 reason_choices = {
     reason_password_failed: _('Username/password check failed'),
-    reason_mfa_failed: _('MFA authentication failed'),
+    reason_mfa_failed: _('MFA failed'),
+    reason_mfa_unset: _('MFA unset'),
     reason_user_not_exist: _("Username does not exist"),
     reason_password_expired: _("Password expired"),
     reason_user_invalid: _('Disabled or expired'),
@@ -46,6 +48,7 @@ block_login_msg = _(
 mfa_failed_msg = _("MFA code invalid, or ntp sync server time")
 
 mfa_required_msg = _("MFA required")
+mfa_unset_msg = _("MFA not set, please set it first")
 login_confirm_required_msg = _("Login confirm required")
 login_confirm_wait_msg = _("Wait login confirm ticket for accept")
 login_confirm_error_msg = _("Login confirm ticket was {}")
@@ -116,6 +119,16 @@ class MFAFailedError(AuthFailedNeedLogMixin, AuthFailedError):
         super().__init__(username=username, request=request)
 
 
+class MFAUnsetError(AuthFailedNeedLogMixin, AuthFailedError):
+    error = reason_mfa_unset
+    msg = mfa_unset_msg
+
+    def __init__(self, user, request, url):
+        super().__init__(username=user.username, request=request)
+        self.user = user
+        self.url = url
+
+
 class BlockLoginError(AuthFailedNeedBlockMixin, AuthFailedError):
     error = 'block_login'
 

apps/authentication/mixins.py

@@ -6,7 +6,7 @@ from django.conf import settings
 from common.utils import get_object_or_none, get_request_ip, get_logger
 from users.models import User
 from users.utils import (
-    is_block_login, clean_failed_count, increase_login_failed_count,
+    is_block_login, clean_failed_count
 )
 from . import errors
 from .utils import check_user_valid
@@ -91,8 +91,9 @@ class AuthMixin:
             return
         if not user.mfa_enabled:
             return
-        if not user.otp_secret_key and user.mfa_is_otp():
-            return
+        unset, url = user.mfa_enabled_but_not_set()
+        if unset:
+            raise errors.MFAUnsetError(user, self.request, url)
         raise errors.MFARequiredError()
 
     def check_user_mfa(self, code):

apps/authentication/templates/authentication/_mfa_confirm_modal.html

@@ -14,7 +14,7 @@
     <label for="mfa" class="col-sm-2 control-label">{% trans 'MFA' %}</label>
     <div class="col-sm-8">
         <input type="text" id="mfa" class="form-control input-sm" name="mfa">
-        <span id="mfa_error" class="help-block">{% trans "Need otp auth for view auth" %}</span>
+        <span id="mfa_error" class="help-block">{% trans "Need MFA for view auth" %}</span>
     </div>
     <div class="col-sm-2">
         <a class="btn btn-primary btn-sm btn-mfa">{% trans "Confirm" %}</a>

apps/authentication/templates/authentication/login.html

@@ -1,116 +1,67 @@
+{% extends '_base_only_msg_content.html' %}
 {% load static %}
 {% load i18n %}
-<!DOCTYPE html>
-<html>
 
-<head>
-    <meta charset="utf-8">
-    <meta name="viewport" content="width=device-width, initial-scale=1.0">
-    <title>Jumpserver</title>
-    <link rel="shortcut icon" href="{{ FAVICON_URL }}" type="image/x-icon">
-    {% include '_head_css_js.html' %}
-    <link href="{% static "css/jumpserver.css" %}" rel="stylesheet">
-    <script type="text/javascript" src="{% url 'javascript-catalog' %}"></script>
-    <script src="{% static "js/jumpserver.js" %}"></script>
-    <style>
-        .captcha {
-             float: right;
-         }
-    </style>
-</head>
+{% block content_title %}
+    {% trans 'Login' %}
+{% endblock %}
 
-<body class="gray-bg">
-    <div class="loginColumns animated fadeInDown">
-        <div class="row">
-            <div class="col-md-6">
-
-                <h2 class="font-bold" style="text-align: center">{% trans 'Welcome to the Jumpserver open source fortress' %}</h2>
-                <p>
-                    {% trans "The world's first fully open source fortress, using the GNU GPL v2.0 open source protocol, is a professional operation and maintenance audit system in compliance with 4A." %}
-                </p>
-                <p>
-                    {% trans "Developed using Python/Django, following the Web 2.0 specification and equipped with industry-leading Web Terminal solutions, with beautiful interactive interface and good user experience." %}
-                </p>
-                <p>
-                    {% trans 'Distributed architecture is adopted to support multi-machine room deployment across regions, central node provides API, and each machine room deploys login node, which can be extended horizontally and without concurrent access restrictions.' %}
-                </p>
-                <p>
-                    {% trans  "Changes the world, starting with a little bit." %}
-                </p>
+{% block content %}
+    <form class="m-t" role="form" method="post" action="">
+        {% csrf_token %}
+        {% if form.non_field_errors %}
+            <div style="line-height: 17px;">
+                <p class="red-fonts">{{ form.non_field_errors.as_text }}</p>
             </div>
-            <div class="col-md-6">
-                <div class="ibox-content">
-                    <div>
-                        <img src="{{ LOGO_URL }}" width="60" height="60">
-                        <span class="font-bold text-center" style="font-size: 24px; font-family: inherit; margin-left: 20px">{% trans 'Login' %}</span>
-                    </div>
-                    <form class="m-t" role="form" method="post" action="">
-                        {% csrf_token %}
-                        {% if form.non_field_errors %}
-                            <div style="line-height: 17px;">
-                                <p class="red-fonts">{{ form.non_field_errors.as_text }}</p>
-                            </div>
-                        {% elif form.errors.captcha %}
-                            <p class="red-fonts">{% trans 'Captcha invalid' %}</p>
-                        {% endif %}
-
-                        <div class="form-group">
-                            <input type="text" class="form-control" name="{{ form.username.html_name }}" placeholder="{% trans 'Username' %}" required="" value="{% if form.username.value %}{{ form.username.value }}{% endif %}">
-                            {% if form.errors.username %}
-                                <div class="help-block field-error">
-                                    <p class="red-fonts">{{ form.errors.username.as_text }}</p>
-                                </div>
-                            {% endif %}
-                        </div>
-                        <div class="form-group">
-                            <input type="password" class="form-control" name="{{ form.password.html_name }}" placeholder="{% trans 'Password' %}" required="">
-                            {% if form.errors.password %}
-                                <div class="help-block field-error">
-                                    <p class="red-fonts">{{ form.errors.password.as_text }}</p>
-                                </div>
-                            {% endif %}
-                        </div>
-                        <div>
-                            {{ form.captcha }}
-                        </div>
-                        <button type="submit" class="btn btn-primary block full-width m-b">{% trans 'Login' %}</button>
-
-                        {% if demo_mode %}
-                        <p class="text-muted font-bold" style="color: red">
-                            Demo账号: admin 密码: admin
-                        </p>
-                        {% endif %}
-
-                        <div class="text-muted text-center">
-                            <div>
-                                <a href="{% url 'users:forgot-password' %}">
-                                    <small>{% trans 'Forgot password' %}?</small>
-                                </a>
-                            </div>
-                        </div>
+        {% elif form.errors.captcha %}
+            <p class="red-fonts">{% trans 'Captcha invalid' %}</p>
+        {% endif %}
 
-                        {% if AUTH_OPENID %}
-                        <div class="hr-line-dashed"></div>
-                        <p class="text-muted text-center">{% trans "More login options" %}</p>
-                        <div>
-                            <button type="button" class="btn btn-default btn-sm btn-block" onclick="location.href='{% url 'authentication:openid:openid-login' %}'">
-                                <i class="fa fa-openid"></i>
-                                {% trans 'Keycloak' %}
-                            </button>
-                        </div>
-                        {% endif %}
+        <div class="form-group">
+            <input type="text" class="form-control" name="{{ form.username.html_name }}" placeholder="{% trans 'Username' %}" required="" value="{% if form.username.value %}{{ form.username.value }}{% endif %}">
+            {% if form.errors.username %}
+                <div class="help-block field-error">
+                    <p class="red-fonts">{{ form.errors.username.as_text }}</p>
+                </div>
+            {% endif %}
+        </div>
+        <div class="form-group">
+            <input type="password" class="form-control" name="{{ form.password.html_name }}" placeholder="{% trans 'Password' %}" required="">
+            {% if form.errors.password %}
+                <div class="help-block field-error">
+                    <p class="red-fonts">{{ form.errors.password.as_text }}</p>
+                </div>
+            {% endif %}
+        </div>
+        <div>
+            {{ form.captcha }}
+        </div>
+        <button type="submit" class="btn btn-primary block full-width m-b">{% trans 'Login' %}</button>
 
-                    </form>
+        {% if demo_mode %}
+            <p class="text-muted font-bold" style="color: red">
+                Demo账号: admin 密码: admin
+            </p>
+        {% endif %}
 
-                </div>
+        <div class="text-muted text-center">
+            <div>
+                <a href="{% url 'users:forgot-password' %}">
+                    <small>{% trans 'Forgot password' %}?</small>
+                </a>
             </div>
         </div>
-        <hr/>
-        <div class="row">
-            <div class="col-md-12">
-                 {% include '_copyright.html' %}
+
+        {% if AUTH_OPENID %}
+            <div class="hr-line-dashed"></div>
+            <p class="text-muted text-center">{% trans "More login options" %}</p>
+            <div>
+                <button type="button" class="btn btn-default btn-sm btn-block" onclick="location.href='{% url 'authentication:openid:openid-login' %}'">
+                    <i class="fa fa-openid"></i>
+                    {% trans 'Keycloak' %}
+                </button>
             </div>
-        </div>
-    </div>
-</body>
-</html>
+        {% endif %}
+
+    </form>
+{% endblock %}

apps/authentication/templates/authentication/login_otp.html

@@ -1,88 +1,32 @@
+{% extends '_base_only_content.html' %}
 {% load static %}
 {% load i18n %}
-<!DOCTYPE html>
-<html>
 
-<head>
-    <meta charset="utf-8">
-    <meta name="viewport" content="width=device-width, initial-scale=1.0">
-    <title>  {{ JMS_TITLE }} </title>
-    <link rel="shortcut icon" href="{{ FAVICON_URL }}" type="image/x-icon">
-    {% include '_head_css_js.html' %}
-    <link href="{% static "css/jumpserver.css" %}" rel="stylesheet">
-    <script type="text/javascript" src="{% url 'javascript-catalog' %}"></script>
-    <script src="{% static "js/jumpserver.js" %}"></script>
-    <script src="{% static "js/plugins/qrcode/qrcode.min.js" %}"></script>
-    <style>
-        .captcha {
-             float: right;
-         }
-    </style>
-</head>
-
-<body class="gray-bg">
-    <div class="loginColumns animated fadeInDown">
-        <div class="row">
-            <div class="col-md-6">
-                <h2 class="font-bold">{% trans 'Welcome to the Jumpserver open source fortress' %}</h2>
-                <p>
-                    {% trans "The world's first fully open source fortress, using the GNU GPL v2.0 open source protocol, is a professional operation and maintenance audit system in compliance with 4A." %}
-                </p>
-                <p>
-                    {% trans "Developed using Python/Django, following the Web 2.0 specification and equipped with industry-leading Web Terminal solutions, with beautiful interactive interface and good user experience." %}
-                </p>
-                <p>
-                    {% trans 'Distributed architecture is adopted to support multi-machine room deployment across regions, central node provides API, and each machine room deploys login node, which can be extended horizontally and without concurrent access restrictions.' %}
-                </p>
-                <p>
-                    {% trans  "Changes the world, starting with a little bit." %}
-                </p>
-
-            </div>
-            <div class="col-md-6">
-                <div class="ibox-content">
-                    <div>
-                        <img src="{% static 'img/logo.png' %}" width="60" height="60">
-                        <span class="font-bold text-center" style="font-size: 24px; font-family: inherit; margin-left: 20px">{% trans 'MFA certification' %}</span>
-                    </div>
-                    <div  class="m-t">
-
-                        <div class="form-group">
-                            <p style="margin:30px auto;" class="text-center"><strong style="color:#000000">{% trans 'The account protection has been opened, please complete the following operations according to the prompts' %}</strong></p>
-                            <div class="text-center">
-                                <img src="{% static 'img/otp_auth.png' %}" alt="" width="72px" height="117">
-                            </div>
-                            <p style="margin: 30px auto">&nbsp;{% trans 'Open Authenticator and enter the 6-bit dynamic code' %}</p>
-                        </div>
-
-                        <form class="m-t" role="form" method="post" action="">
-
-                            {% csrf_token %}
-                            {% if 'otp_code' in form.errors %}
-                                <p class="red-fonts">{{ form.otp_code.errors.as_text }}</p>
-                            {% endif %}
-                            <div class="form-group">
-                                <input type="text" class="form-control" name="otp_code" placeholder="{% trans 'Six figures' %}" required="">
-                            </div>
-                            <button type="submit" class="btn btn-primary block full-width m-b">{% trans 'Next' %}</button>
-
-                            <a href="#">
-                                <small>{% trans "Can't provide security? Please contact the administrator!" %}</small>
-                            </a>
-
-                        </form>
-                    </div>
-                    <p class="m-t">
-                    </p>
-                </div>
-            </div>
+{% block title %}
+    {% trans 'MFA' %}
+{% endblock %}
+
+{% block content %}
+    <form class="m-t" role="form" method="post" action="">
+        {% csrf_token %}
+        {% if 'otp_code' in form.errors %}
+            <p class="red-fonts">{{ form.otp_code.errors.as_text }}</p>
+        {% endif %}
+        <div class="form-group">
+            <select class="form-control">
+                <option value="otp" selected>{% trans 'One-time password' %}</option>
+            </select>
         </div>
-        <hr/>
-        <div class="row">
-            <div class="col-md-12">
-                 {% include '_copyright.html' %}
-            </div>
+        <div class="form-group">
+            <input type="text" class="form-control" name="otp_code" placeholder="" required="" autofocus="autofocus">
+            <span class="help-block">
+                {% trans 'Open Google Authenticator and enter the 6-bit dynamic code' %}
+            </span>
+        </div>
+        <button type="submit" class="btn btn-primary block full-width m-b">{% trans 'Next' %}</button>
+
+        <div>
+            <small>{% trans "Can't provide security? Please contact the administrator!" %}</small>
         </div>
-    </div>
-</body>
-</html>
+    </form>
+{% endblock %}

apps/authentication/urls/view_urls.py

@@ -1,8 +1,6 @@
 # coding:utf-8
 #
 
-from __future__ import absolute_import
-
 from django.urls import path, include
 
 from .. import views
@@ -10,13 +8,14 @@ from .. import views
 app_name = 'authentication'
 
 urlpatterns = [
-    # openid
-    path('openid/', include(('authentication.backends.openid.urls', 'authentication'), namespace='openid')),
-
     # login
     path('login/', views.UserLoginView.as_view(), name='login'),
     path('login/otp/', views.UserLoginOtpView.as_view(), name='login-otp'),
     path('login/wait-confirm/', views.UserLoginWaitConfirmView.as_view(), name='login-wait-confirm'),
     path('login/guard/', views.UserLoginGuardView.as_view(), name='login-guard'),
     path('logout/', views.UserLogoutView.as_view(), name='logout'),
+
+    # openid
+    path('openid/', include(('authentication.backends.openid.urls', 'authentication'), namespace='openid')),
+    path('cas/', include(('authentication.backends.cas.urls', 'authentication'), namespace='cas')),
 ]

apps/authentication/views/login.py

@@ -20,7 +20,7 @@ from django.urls import reverse_lazy
 
 from common.utils import get_request_ip, get_object_or_none
 from users.utils import (
-    redirect_user_first_login_or_index, set_tmp_user_to_cache
+    redirect_user_first_login_or_index
 )
 from .. import forms, mixins, errors
 
@@ -52,17 +52,29 @@ class UserLoginView(mixins.AuthMixin, FormView):
         template_name = 'authentication/xpack_login.html'
         return template_name
 
+    def get_redirect_url_if_need(self, request):
+        redirect_url = ''
+        # show jumpserver login page if request http://{JUMP-SERVER}/?admin=1
+        if self.request.GET.get("admin", 0):
+            return None
+        if settings.AUTH_OPENID:
+            redirect_url = reverse("authentication:openid:openid-login")
+        elif settings.AUTH_CAS:
+            redirect_url = reverse(settings.CAS_LOGIN_URL_NAME)
+
+        if redirect_url:
+            query_string = request.GET.urlencode()
+            redirect_url = "{}?{}".format(redirect_url, query_string)
+        return redirect_url
+
     def get(self, request, *args, **kwargs):
         if request.user.is_staff:
             return redirect(redirect_user_first_login_or_index(
                 request, self.redirect_field_name)
             )
-        # show jumpserver login page if request http://{JUMP-SERVER}/?admin=1
-        if settings.AUTH_OPENID and not self.request.GET.get('admin', 0):
-            query_string = request.GET.urlencode()
-            openid_login_url = reverse_lazy("authentication:openid:openid-login")
-            login_url = "{}?{}".format(openid_login_url, query_string)
-            return redirect(login_url)
+        redirect_url = self.get_redirect_url_if_need(request)
+        if redirect_url:
+            return redirect(redirect_url)
         request.session.set_test_cookie()
         return super().get(request, *args, **kwargs)
 
@@ -127,12 +139,9 @@ class UserLoginGuardView(mixins.AuthMixin, RedirectView):
             return self.format_redirect_url(self.login_otp_url)
         except errors.LoginConfirmBaseError:
             return self.format_redirect_url(self.login_confirm_url)
+        except errors.MFAUnsetError as e:
+            return e.url
         else:
-            # 启用但是没有设置otp, 排除radius
-            if user.mfa_enabled_but_not_set():
-                # 1,2,mfa_setting & F
-                set_tmp_user_to_cache(self.request, user)
-                return reverse('users:user-otp-enable-authentication')
             auth_login(self.request, user)
             self.send_auth_signal(success=True, user=user)
             self.clear_auth_mark()
@@ -174,8 +183,17 @@ class UserLoginWaitConfirmView(TemplateView):
 class UserLogoutView(TemplateView):
     template_name = 'flash_message_standalone.html'
 
+    @staticmethod
+    def get_backend_logout_url():
+        if settings.AUTH_CAS:
+            return settings.CAS_LOGOUT_URL_NAME
+        return None
+
     def get(self, request, *args, **kwargs):
         auth_logout(request)
+        backend_logout_url = self.get_backend_logout_url()
+        if backend_logout_url:
+            return redirect(backend_logout_url)
         next_uri = request.COOKIES.get("next")
         if next_uri:
             return redirect(next_uri)

apps/common/README.md

@@ -16,7 +16,7 @@ provide this ability, not common, You should write it on your app utils.
 ## Celery usage 
 
 
-Jumpserver use celery to run task async. Using redis as the broker, so
+JumpServer use celery to run task async. Using redis as the broker, so
 you should run a redis instance
 
 #### Run redis

apps/common/mixins/api.py

@@ -1,13 +1,20 @@
 # -*- coding: utf-8 -*-
 #
+import time
+from hashlib import md5
+from threading import Thread
+
+from django.core.cache import cache
 from django.http import JsonResponse
+from rest_framework.response import Response
 from rest_framework.settings import api_settings
 
 from common.drf.filters import IDSpmFilter, CustomFilter
+from ..utils import lazyproperty
 
 __all__ = [
     "JSONResponseMixin", "CommonApiMixin",
-    "IDSpmFilterMixin",
+    "IDSpmFilterMixin", 'AsyncApiMixin',
 ]
 
 
@@ -62,3 +69,122 @@ class ExtraFilterFieldsMixin:
 
 class CommonApiMixin(SerializerMixin, ExtraFilterFieldsMixin):
     pass
+
+
+class InterceptMixin:
+    def dispatch(self, request, *args, **kwargs):
+        self.args = args
+        self.kwargs = kwargs
+        request = self.initialize_request(request, *args, **kwargs)
+        self.request = request
+        self.headers = self.default_response_headers  # deprecate?
+
+        try:
+            self.initial(request, *args, **kwargs)
+
+            # Get the appropriate handler method
+            if request.method.lower() in self.http_method_names:
+                handler = getattr(self, request.method.lower(),
+                                  self.http_method_not_allowed)
+            else:
+                handler = self.http_method_not_allowed
+
+            response = self.do(handler, request, *args, **kwargs)
+
+        except Exception as exc:
+            response = self.handle_exception(exc)
+
+        self.response = self.finalize_response(request, response, *args, **kwargs)
+        return self.response
+
+
+class AsyncApiMixin(InterceptMixin):
+    def get_request_user_id(self):
+        user = self.request.user
+        if hasattr(user, 'id'):
+            return str(user.id)
+        return ''
+
+    @lazyproperty
+    def async_cache_key(self):
+        method = self.request.method
+        path = self.get_request_md5()
+        user = self.get_request_user_id()
+        key = '{}_{}_{}'.format(method, path, user)
+        return key
+
+    def get_request_md5(self):
+        path = self.request.path
+        query = {k: v for k, v in self.request.GET.items()}
+        query.pop("_", None)
+        query.pop('refresh', None)
+        query = "&".join(["{}={}".format(k, v) for k, v in query.items()])
+        full_path = "{}?{}".format(path, query)
+        return md5(full_path.encode()).hexdigest()
+
+    @lazyproperty
+    def initial_data(self):
+        data = {
+            "status": "running",
+            "start_time": time.time(),
+            "key": self.async_cache_key,
+        }
+        return data
+
+    def get_cache_data(self):
+        key = self.async_cache_key
+        if self.is_need_refresh():
+            cache.delete(key)
+            return None
+        data = cache.get(key)
+        return data
+
+    def do(self, handler, *args, **kwargs):
+        if not self.is_need_async():
+            return handler(*args, **kwargs)
+        resp = self.do_async(handler, *args, **kwargs)
+        return resp
+
+    def is_need_refresh(self):
+        if self.request.GET.get("refresh"):
+            return True
+        return False
+
+    def is_need_async(self):
+        return False
+
+    def do_async(self, handler, *args, **kwargs):
+        data = self.get_cache_data()
+        if not data:
+            t = Thread(
+                target=self.do_in_thread,
+                args=(handler, *args),
+                kwargs=kwargs
+            )
+            t.start()
+            resp = Response(self.initial_data)
+            return resp
+        status = data.get("status")
+        resp = data.get("resp")
+        if status == "ok" and resp:
+            resp = Response(**resp)
+        else:
+            resp = Response(data)
+        return resp
+
+    def do_in_thread(self, handler, *args, **kwargs):
+        key = self.async_cache_key
+        data = self.initial_data
+        cache.set(key, data, 600)
+        try:
+            response = handler(*args, **kwargs)
+            data["status"] = "ok"
+            data["resp"] = {
+                "data": response.data,
+                "status": response.status_code
+            }
+            cache.set(key, data, 600)
+        except Exception as e:
+            data["error"] = str(e)
+            data["status"] = "error"
+            cache.set(key, data, 600)

apps/common/mixins/models.py

@@ -5,7 +5,6 @@ from django.db import models
 from django.utils import timezone
 from django.utils.translation import ugettext_lazy as _
 
-
 __all__ = [
     "NoDeleteManager", "NoDeleteModelMixin", "NoDeleteQuerySet",
     "CommonModelMixin"
@@ -65,3 +64,5 @@ class DebugQueryManager(models.Manager):
         print("<<<<<<<<<<<<<<<<<<<<<<<<<<<<")
         queryset = super().get_queryset()
         return queryset
+
+

apps/common/struct.py

@@ -1,5 +1,7 @@
 # -*- coding: utf-8 -*-
 #
+from itertools import chain
+from .utils import lazyproperty
 
 
 class Stack(list):
@@ -23,3 +25,82 @@ class Stack(list):
 
     def push(self, item):
         self.append(item)
+
+
+class QuerySetChain:
+    def __init__(self, querysets):
+        self.querysets = querysets
+
+    @lazyproperty
+    def querysets_counts(self):
+        counts = [s.count() for s in self.querysets]
+        return counts
+
+    def count(self):
+        return self.total_count
+
+    @lazyproperty
+    def total_count(self):
+        return sum(self.querysets_counts)
+
+    def __iter__(self):
+        self._chain = chain(*self.querysets)
+        return self
+
+    def __next__(self):
+        return next(self._chain)
+
+    def __getitem__(self, ndx):
+        querysets_count_zip = zip(self.querysets, self.querysets_counts)
+        length = 0   # 加上本数组后的大数组长度
+        pre_length = 0  # 不包含本数组的大数组长度
+        items = []  # 返回的值
+        loop = 0
+
+        if isinstance(ndx, slice):
+            ndx_start = ndx.start or 0
+            ndx_stop = ndx.stop or self.total_count
+            ndx_step = ndx.step or 1
+        else:
+            ndx_start = ndx
+            ndx_stop, ndx_step = None, None
+
+        for queryset, count in querysets_count_zip:
+            length += count
+            loop += 1
+            # 取当前数组的start角标, 存在3中情况
+            # 1. start角标在当前数组
+            if length > ndx_start >= pre_length:
+                start = ndx_start - pre_length
+                # print("[loop {}] Start is: {}".format(loop, start))
+                if ndx_step is None:
+                    return queryset[start]
+            # 2. 不包含当前数组，因为起始已经超过了当前数组的长度
+            elif ndx_start >= length:
+                pre_length += count
+                continue
+            # 3. 不在当前数组，但是应该从当前数组0开始计算
+            else:
+                start = 0
+
+            # 可能取单个值, ndx_stop 为None, 不应该再找
+            if ndx_stop is None:
+                pre_length += count
+                continue
+
+            # 取当前数组的stop角标, 存在2中情况
+            # 不存在第3中情况是因为找到了会提交结束循环
+            # 1. 结束角标小于length代表 结束位在当前数组上
+            if ndx_stop < length:
+                stop = ndx_stop - pre_length
+            # 2. 结束位置包含改数组到了最后
+            else:
+                stop = count
+            # print("[loop {}] Slice: {} {} {}".format(loop, start, stop, ndx_step))
+            items.extend(list(queryset[slice(start, stop, ndx_step)]))
+            pre_length += count
+
+            # 如果结束再当前数组，则结束循环
+            if ndx_stop < length:
+                break
+        return items

apps/common/utils/common.py

@@ -199,11 +199,15 @@ logger = get_logger(__name__)
 
 def timeit(func):
     def wrapper(*args, **kwargs):
-        logger.debug("Start call: {}".format(func.__name__))
+        if hasattr(func, '__name__'):
+            name = func.__name__
+        else:
+            name = func
+        logger.debug("Start call: {}".format(name))
         now = time.time()
         result = func(*args, **kwargs)
         using = (time.time() - now) * 1000
-        msg = "End call {}, using: {:.1f}ms".format(func.__name__, using)
+        msg = "End call {}, using: {:.1f}ms".format(name, using)
         logger.debug(msg)
         return result
     return wrapper

apps/common/utils/random.py

@@ -0,0 +1,39 @@
+# -*- coding: utf-8 -*-
+#
+import socket
+import struct
+import random
+
+
+def random_datetime(date_start, date_end):
+    random_delta = (date_end - date_start) * random.random()
+    return date_start + random_delta
+
+
+def random_ip():
+    return socket.inet_ntoa(struct.pack('>I', random.randint(1, 0xffffffff)))
+
+
+
+# def strTimeProp(start, end, prop, fmt):
+#     time_start = time.mktime(time.strptime(start, fmt))
+#     time_end = time.mktime(time.strptime(end, fmt))
+#     ptime = time_start + prop * (time_end - time_start)
+#     return int(ptime)
+#
+#
+# def randomTimestamp(start, end, fmt='%Y-%m-%d %H:%M:%S'):
+#     return strTimeProp(start, end, random.random(), fmt)
+#
+#
+# def randomDate(start, end, frmt='%Y-%m-%d %H:%M:%S'):
+#     return time.strftime(frmt, time.localtime(strTimeProp(start, end, random.random(), frmt)))
+#
+#
+# def randomTimestampList(start, end, n, frmt='%Y-%m-%d %H:%M:%S'):
+#     return [randomTimestamp(start, end, frmt) for _ in range(n)]
+#
+#
+# def randomDateList(start, end, n, frmt='%Y-%m-%d %H:%M:%S'):
+#     return [randomDate(start, end, frmt) for _ in range(n)]
+

apps/jumpserver/conf.py

@@ -84,11 +84,10 @@ class Config(dict):
     :param defaults: an optional dictionary of default values
     """
     defaults = {
-        # Django Config
+        # Django Config, Must set before start
         'SECRET_KEY': '',
         'BOOTSTRAP_TOKEN': '',
         'DEBUG': True,
-        'SITE_URL': 'http://localhost:8080',
         'LOG_LEVEL': 'DEBUG',
         'LOG_DIR': os.path.join(PROJECT_DIR, 'logs'),
         'DB_ENGINE': 'mysql',
@@ -100,10 +99,13 @@ class Config(dict):
         'REDIS_HOST': '127.0.0.1',
         'REDIS_PORT': 6379,
         'REDIS_PASSWORD': '',
+        # Default value
         'REDIS_DB_CELERY': 3,
         'REDIS_DB_CACHE': 4,
         'REDIS_DB_SESSION': 5,
         'REDIS_DB_WS': 6,
+
+        'SITE_URL': 'http://localhost:8080',
         'CAPTCHA_TEST_MODE': None,
         'TOKEN_EXPIRATION': 3600 * 24,
         'DISPLAY_PER_PAGE': 25,
@@ -148,8 +150,13 @@ class Config(dict):
         'RADIUS_ENCRYPT_PASSWORD': True,
         'OTP_IN_RADIUS': False,
 
+        'AUTH_CAS': False,
+        'CAS_SERVER_URL': "http://host/cas/",
+        'CAS_LOGOUT_COMPLETELY': False,
+        'CAS_VERSION': 3,
+
         'OTP_VALID_WINDOW': 2,
-        'OTP_ISSUER_NAME': 'Jumpserver',
+        'OTP_ISSUER_NAME': 'JumpServer',
         'EMAIL_SUFFIX': 'jumpserver.org',
 
         'TERMINAL_PASSWORD_AUTH': True,
@@ -179,6 +186,7 @@ class Config(dict):
         'HTTP_LISTEN_PORT': 8080,
         'WS_LISTEN_PORT': 8070,
         'LOGIN_LOG_KEEP_DAYS': 90,
+        'TASK_LOG_KEEP_DAYS': 10,
         'ASSETS_PERM_CACHE_TIME': 3600 * 24,
         'SECURITY_MFA_VERIFY_TTL': 3600,
         'ASSETS_PERM_CACHE_ENABLE': False,
@@ -284,6 +292,8 @@ class DynamicConfig:
         ]
         if self.get('AUTH_LDAP'):
             backends.insert(0, 'authentication.backends.ldap.LDAPAuthorizationBackend')
+        if self.static_config.get('AUTH_CAS'):
+            backends.insert(0, 'authentication.backends.cas.CASBackend')
         if self.static_config.get('AUTH_OPENID'):
             backends.insert(0, 'authentication.backends.openid.backends.OpenIDAuthorizationPasswordBackend')
             backends.insert(0, 'authentication.backends.openid.backends.OpenIDAuthorizationCodeBackend')

apps/jumpserver/context_processor.py

@@ -13,7 +13,7 @@ def jumpserver_processor(request):
         'LOGO_TEXT_URL': static('img/logo_text.png'),
         'LOGIN_IMAGE_URL': static('img/login_image.png'),
         'FAVICON_URL': static('img/facio.ico'),
-        'JMS_TITLE': 'Jumpserver',
+        'JMS_TITLE': 'JumpServer',
         'VERSION': settings.VERSION,
         'COPYRIGHT': 'FIT2CLOUD 飞致云' + ' © 2014-2020',
         'SECURITY_COMMAND_EXECUTION': settings.SECURITY_COMMAND_EXECUTION,

apps/jumpserver/settings/auth.py

@@ -64,7 +64,21 @@ RADIUS_SERVER = CONFIG.RADIUS_SERVER
 RADIUS_PORT = CONFIG.RADIUS_PORT
 RADIUS_SECRET = CONFIG.RADIUS_SECRET
 
+# CAS Auth
+AUTH_CAS = CONFIG.AUTH_CAS
+CAS_SERVER_URL = CONFIG.CAS_SERVER_URL
+CAS_VERIFY_SSL_CERTIFICATE = False
+CAS_LOGIN_URL_NAME = "authentication:cas:cas-login"
+CAS_LOGOUT_URL_NAME = "authentication:cas:cas-logout"
+CAS_LOGIN_MSG = None
+CAS_LOGGED_MSG = None
+CAS_LOGOUT_COMPLETELY = CONFIG.CAS_LOGOUT_COMPLETELY
+CAS_VERSION = CONFIG.CAS_VERSION
 
+
+# Other setting
 TOKEN_EXPIRATION = CONFIG.TOKEN_EXPIRATION
 LOGIN_CONFIRM_ENABLE = CONFIG.LOGIN_CONFIRM_ENABLE
 OTP_IN_RADIUS = CONFIG.OTP_IN_RADIUS
+
+AUTHENTICATION_BACKENDS = DYNAMIC.AUTHENTICATION_BACKENDS

apps/jumpserver/settings/base.py

@@ -51,6 +51,7 @@ INSTALLED_APPS = [
     'rest_framework',
     'rest_framework_swagger',
     'drf_yasg',
+    'django_cas_ng',
     'channels',
     'django_filters',
     'bootstrap3',
@@ -75,6 +76,7 @@ MIDDLEWARE = [
     'django.contrib.messages.middleware.MessageMiddleware',
     'django.middleware.clickjacking.XFrameOptionsMiddleware',
     'authentication.backends.openid.middleware.OpenIDAuthenticationMiddleware',
+    'django_cas_ng.middleware.CASMiddleware',
     'jumpserver.middleware.TimezoneMiddleware',
     'jumpserver.middleware.DemoMiddleware',
     'jumpserver.middleware.RequestMiddleware',
@@ -220,8 +222,6 @@ EMAIL_USE_SSL = DYNAMIC.EMAIL_USE_SSL
 EMAIL_USE_TLS = DYNAMIC.EMAIL_USE_TLS
 
 
-AUTHENTICATION_BACKENDS = DYNAMIC.AUTHENTICATION_BACKENDS
-
 # Custom User Auth model
 AUTH_USER_MODEL = 'users.User'
 

apps/jumpserver/settings/custom.py

@@ -82,5 +82,6 @@ USER_GUIDE_URL = DYNAMIC.USER_GUIDE_URL
 HTTP_LISTEN_PORT = CONFIG.HTTP_LISTEN_PORT
 WS_LISTEN_PORT = CONFIG.WS_LISTEN_PORT
 LOGIN_LOG_KEEP_DAYS = DYNAMIC.LOGIN_LOG_KEEP_DAYS
+TASK_LOG_KEEP_DAYS = CONFIG.TASK_LOG_KEEP_DAYS
 ORG_CHANGE_TO_URL = CONFIG.ORG_CHANGE_TO_URL
 WINDOWS_SKIP_ALL_MANUAL_PASSWORD = CONFIG.WINDOWS_SKIP_ALL_MANUAL_PASSWORD

apps/jumpserver/settings/libs.py

@@ -53,6 +53,7 @@ SWAGGER_SETTINGS = {
             'in': 'header'
         }
     },
+    'DEFAULT_INFO': 'jumpserver.views.swagger.api_info',
 }
 
 

apps/jumpserver/settings/logging.py

@@ -96,12 +96,16 @@ LOGGING = {
             'handlers': ['syslog'],
             'level': 'INFO'
         },
-        # 'django.db': {
-        #     'handlers': ['console', 'file'],
-        #     'level': 'DEBUG'
-        # }
+
     }
 }
+
+if os.environ.get("DEBUG_DB"):
+    LOGGING['loggers']['django.db'] = {
+       'handlers': ['console', 'file'],
+       'level': 'DEBUG'
+    }
+
 SYSLOG_ENABLE = CONFIG.SYSLOG_ENABLE
 
 if CONFIG.SYSLOG_ADDR != '' and len(CONFIG.SYSLOG_ADDR.split(':')) == 2:

apps/jumpserver/views/index.py

@@ -1,182 +1,246 @@
-import datetime
-
+from django.core.cache import cache
 from django.views.generic import TemplateView
 from django.utils import timezone
 from django.utils.translation import ugettext_lazy as _
-from django.db.models import Count
+from django.db.models import Count, Max
 from django.shortcuts import redirect
 
-
 from users.models import User
 from assets.models import Asset
 from terminal.models import Session
 from orgs.utils import current_org
 from common.permissions import PermissionsMixin, IsValidUser
+from common.utils import timeit, lazyproperty
 
 __all__ = ['IndexView']
 
 
-class IndexView(PermissionsMixin, TemplateView):
-    template_name = 'index.html'
-    permission_classes = [IsValidUser]
-
-    session_week = None
-    session_month = None
-    session_month_dates = []
-    session_month_dates_archive = []
-
-    def dispatch(self, request, *args, **kwargs):
-        if not request.user.is_authenticated:
-            return self.handle_no_permission()
-        if request.user.is_common_user:
-            return redirect('assets:user-asset-list')
-        return super(IndexView, self).dispatch(request, *args, **kwargs)
-
-    @staticmethod
-    def get_user_count():
-        return current_org.get_org_members().count()
+class MonthLoginMetricMixin:
+    @lazyproperty
+    def session_month(self):
+        month_ago = timezone.now() - timezone.timedelta(days=30)
+        session_month = Session.objects.filter(date_start__gt=month_ago)
+        return session_month
 
-    @staticmethod
-    def get_asset_count():
-        return Asset.objects.all().count()
+    @lazyproperty
+    def session_month_dates(self):
+        return self.session_month.dates('date_start', 'day')
 
-    @staticmethod
-    def get_online_user_count():
-        return len(set(Session.objects.filter(is_finished=False).values_list('user', flat=True)))
+    def get_month_day_metrics(self):
+        month_str = [
+            d.strftime('%m-%d') for d in self.session_month_dates
+        ] or ['0']
+        return month_str
 
     @staticmethod
-    def get_online_session_count():
-        return Session.objects.filter(is_finished=False).count()
+    def get_cache_key(date, tp):
+        date_str = date.strftime("%Y%m%d")
+        key = "SESSION_MONTH_{}_{}".format(tp, date_str)
+        return key
+
+    def __get_data_from_cache(self, date, tp):
+        if date == timezone.now().date():
+            return None
+        cache_key = self.get_cache_key(date, tp)
+        count = cache.get(cache_key)
+        return count
 
-    def get_top5_user_a_week(self):
-        return self.session_week.values('user').annotate(total=Count('user')).order_by('-total')[:5]
+    def __set_data_to_cache(self, date, tp, count):
+        cache_key = self.get_cache_key(date, tp)
+        cache.set(cache_key, count, 3600*24*7)
 
-    def get_week_login_user_count(self):
-        return self.session_week.values('user').distinct().count()
+    @lazyproperty
+    def user_disabled_total(self):
+        return current_org.get_org_members().filter(is_active=False).count()
 
-    def get_week_login_asset_count(self):
-        return self.session_week.count()
+    @lazyproperty
+    def asset_disabled_total(self):
+        return Asset.objects.filter(is_active=False).count()
 
-    def get_month_day_metrics(self):
-        month_str = [d.strftime('%m-%d') for d in self.session_month_dates] or ['0']
-        return month_str
+    def get_date_login_count(self, date):
+        tp = "LOGIN"
+        count = self.__get_data_from_cache(date, tp)
+        if count is not None:
+            return count
+        count = Session.objects.filter(date_start__date=date).count()
+        self.__set_data_to_cache(date, tp, count)
+        return count
 
     def get_month_login_metrics(self):
         data = []
-        time_min = datetime.datetime.min.time()
-        time_max = datetime.datetime.max.time()
         for d in self.session_month_dates:
-            ds = datetime.datetime.combine(d, time_min).replace(tzinfo=timezone.get_current_timezone())
-            de = datetime.datetime.combine(d, time_max).replace(tzinfo=timezone.get_current_timezone())
-            data.append(self.session_month.filter(date_start__range=(ds, de)).count())
+            count = self.get_date_login_count(d)
+            data.append(count)
+        if len(data) == 0:
+            data = [0]
         return data
 
+    def get_date_user_count(self, date):
+        tp = "USER"
+        count = self.__get_data_from_cache(date, tp)
+        if count is not None:
+            return count
+        count = Session.objects.filter(date_start__date=date)\
+            .values('user').distinct().count()
+        self.__set_data_to_cache(date, tp, count)
+        return count
+
     def get_month_active_user_metrics(self):
-        if self.session_month_dates_archive:
-            return [q.values('user').distinct().count()
-                    for q in self.session_month_dates_archive]
-        else:
-            return [0]
+        data = []
+        for d in self.session_month_dates:
+            count = self.get_date_user_count(d)
+            data.append(count)
+        return data
+
+    def get_date_asset_count(self, date):
+        tp = "ASSET"
+        count = self.__get_data_from_cache(date, tp)
+        if count is not None:
+            return count
+        count = Session.objects.filter(date_start__date=date) \
+            .values('asset').distinct().count()
+        self.__set_data_to_cache(date, tp, count)
+        return count
 
     def get_month_active_asset_metrics(self):
-        if self.session_month_dates_archive:
-            return [q.values('asset').distinct().count()
-                    for q in self.session_month_dates_archive]
-        else:
-            return [0]
+        data = []
+        for d in self.session_month_dates:
+            count = self.get_date_asset_count(d)
+            data.append(count)
+        return data
 
-    def get_month_active_user_total(self):
-        return self.session_month.values('user').distinct().count()
+    @lazyproperty
+    def month_active_user_total(self):
+        count = self.session_month.values('user').distinct().count()
+        return count
 
-    def get_month_inactive_user_total(self):
-        count = current_org.get_org_members().count() - self.get_month_active_user_total()
+    @lazyproperty
+    def month_inactive_user_total(self):
+        total = current_org.get_org_members().count()
+        active = self.month_active_user_total
+        count = total - active
         if count < 0:
             count = 0
         return count
 
-    def get_month_active_asset_total(self):
+    @lazyproperty
+    def month_active_asset_total(self):
         return self.session_month.values('asset').distinct().count()
 
-    def get_month_inactive_asset_total(self):
-        count = Asset.objects.all().count() - self.get_month_active_asset_total()
+    @lazyproperty
+    def month_inactive_asset_total(self):
+        total = Asset.objects.all().count()
+        active = self.month_active_asset_total
+        count = total - active
         if count < 0:
             count = 0
         return count
 
-    @staticmethod
-    def get_user_disabled_total():
-        return current_org.get_org_members().filter(is_active=False).count()
+    def get_context_data(self, **kwargs):
+        context = super().get_context_data(**kwargs)
+        context.update({
+            'month_str': self.get_month_day_metrics(),
+            'month_total_visit_count': self.get_month_login_metrics(),
+            'month_user': self.get_month_active_user_metrics(),
+            'mouth_asset': self.get_month_active_asset_metrics(),
+            'month_user_active': self.month_active_user_total,
+            'month_user_inactive': self.month_inactive_user_total,
+            'month_user_disabled': self.user_disabled_total,
+            'month_asset_active': self.month_active_asset_total,
+            'month_asset_inactive': self.month_inactive_asset_total,
+            'month_asset_disabled': self.asset_disabled_total,
+        })
+        return context
 
-    @staticmethod
-    def get_asset_disabled_total():
-        return Asset.objects.filter(is_active=False).count()
 
-    def get_week_top10_asset(self):
-        assets = list(self.session_week.values('asset').annotate(total=Count('asset')).order_by('-total')[:10])
-        for asset in assets:
-            last_login = self.session_week.filter(asset=asset["asset"]).order_by('date_start').last()
-            asset['last'] = last_login
+class WeekSessionMetricMixin:
+    session_week = None
+
+    @lazyproperty
+    def session_week(self):
+        week_ago = timezone.now() - timezone.timedelta(weeks=1)
+        session_week = Session.objects.filter(date_start__gt=week_ago)
+        return session_week
+
+    def get_top5_user_a_week(self):
+        users = self.session_week.values('user') \
+                    .annotate(total=Count('user')) \
+                    .order_by('-total')[:5]
+        return users
+
+    def get_week_login_user_count(self):
+        return self.session_week.values('user').distinct().count()
+
+    def get_week_login_asset_count(self):
+        return self.session_week.count()
+
+    def get_week_top10_assets(self):
+        assets = self.session_week.values("asset")\
+            .annotate(total=Count("asset"))\
+            .annotate(last=Max("date_start")).order_by("-total")[:10]
         return assets
 
-    def get_week_top10_user(self):
-        users = list(self.session_week.values('user').annotate(
-            total=Count('asset')).order_by('-total')[:10])
-        for user in users:
-            last_login = self.session_week.filter(user=user["user"]).order_by('date_start').last()
-            user['last'] = last_login
+    def get_week_top10_users(self):
+        users = self.session_week.values("user") \
+                     .annotate(total=Count("user")) \
+                     .annotate(last=Max("date_start")).order_by("-total")[:10]
         return users
 
     def get_last10_sessions(self):
         sessions = self.session_week.order_by('-date_start')[:10]
         for session in sessions:
-            try:
-                session.avatar_url = User.objects.get(username=session.user).avatar_url()
-            except User.DoesNotExist:
-                session.avatar_url = User.objects.first().avatar_url()
+            session.avatar_url = User.get_avatar_url("")
         return sessions
 
     def get_context_data(self, **kwargs):
-        week_ago = timezone.now() - timezone.timedelta(weeks=1)
-        month_ago = timezone.now() - timezone.timedelta(days=30)
-        self.session_week = Session.objects.filter(date_start__gt=week_ago)
-        self.session_month = Session.objects.filter(date_start__gt=month_ago)
-        self.session_month_dates = self.session_month.dates('date_start', 'day')
+        context = super().get_context_data(**kwargs)
+        context.update({
+            'user_visit_count_weekly': self.get_week_login_user_count(),
+            'asset_visit_count_weekly': self.get_week_login_asset_count(),
+            'user_visit_count_top_five': self.get_top5_user_a_week(),
+            'last_login_ten': self.get_last10_sessions(),
+            'week_asset_hot_ten': self.get_week_top10_assets(),
+            'week_user_hot_ten': self.get_week_top10_users(),
+        })
+        return context
 
-        self.session_month_dates_archive = []
-        time_min = datetime.datetime.min.time()
-        time_max = datetime.datetime.max.time()
 
-        for d in self.session_month_dates:
-            ds = datetime.datetime.combine(d, time_min).replace(
-                tzinfo=timezone.get_current_timezone())
-            de = datetime.datetime.combine(d, time_max).replace(
-                tzinfo=timezone.get_current_timezone())
-            self.session_month_dates_archive.append(
-                self.session_month.filter(date_start__range=(ds, de)))
-
-        context = {
+class IndexView(PermissionsMixin, MonthLoginMetricMixin, WeekSessionMetricMixin, TemplateView):
+    template_name = 'index.html'
+    permission_classes = [IsValidUser]
+
+    def dispatch(self, request, *args, **kwargs):
+        if not request.user.is_authenticated:
+            return self.handle_no_permission()
+        if request.user.is_common_user:
+            return redirect('assets:user-asset-list')
+        return super(IndexView, self).dispatch(request, *args, **kwargs)
+
+    @staticmethod
+    def get_user_count():
+        return current_org.get_org_members().count()
+
+    @staticmethod
+    def get_asset_count():
+        return Asset.objects.all().count()
+
+    @staticmethod
+    def get_online_user_count():
+        count = Session.objects.filter(is_finished=False)\
+            .values_list('user', flat=True).distinct().count()
+        return count
+
+    @staticmethod
+    def get_online_session_count():
+        return Session.objects.filter(is_finished=False).count()
+
+    def get_context_data(self, **kwargs):
+        context = super().get_context_data(**kwargs)
+        context.update({
             'assets_count': self.get_asset_count(),
             'users_count': self.get_user_count(),
             'online_user_count': self.get_online_user_count(),
             'online_asset_count': self.get_online_session_count(),
-            'user_visit_count_weekly': self.get_week_login_user_count(),
-            'asset_visit_count_weekly': self.get_week_login_asset_count(),
-            'user_visit_count_top_five': self.get_top5_user_a_week(),
-            'month_str': self.get_month_day_metrics(),
-            'month_total_visit_count': self.get_month_login_metrics(),
-            'month_user': self.get_month_active_user_metrics(),
-            'mouth_asset': self.get_month_active_asset_metrics(),
-            'month_user_active': self.get_month_active_user_total(),
-            'month_user_inactive': self.get_month_inactive_user_total(),
-            'month_user_disabled': self.get_user_disabled_total(),
-            'month_asset_active': self.get_month_active_asset_total(),
-            'month_asset_inactive': self.get_month_inactive_asset_total(),
-            'month_asset_disabled': self.get_asset_disabled_total(),
-            'week_asset_hot_ten': self.get_week_top10_asset(),
-            'last_login_ten': self.get_last10_sessions(),
-            'week_user_hot_ten': self.get_week_top10_user(),
             'app': _("Dashboard"),
-        }
-
-        kwargs.update(context)
-        return super(IndexView, self).get_context_data(**kwargs)
+        })
+        return context

apps/jumpserver/views/swagger.py

@@ -49,6 +49,16 @@ class CustomSwaggerAutoSchema(SwaggerAutoSchema):
         return fields
 
 
+api_info = openapi.Info(
+    title="JumpServer API Docs",
+    default_version='v1',
+    description="JumpServer Restful api docs",
+    terms_of_service="https://www.jumpserver.org",
+    contact=openapi.Contact(email="support@fit2cloud.com"),
+    license=openapi.License(name="GPLv2 License"),
+)
+
+
 def get_swagger_view(version='v1'):
     from ..urls import api_v1, api_v2
     from django.urls import path, include
@@ -65,14 +75,7 @@ def get_swagger_view(version='v1'):
     else:
         patterns = api_v1_patterns
     schema_view = get_schema_view(
-        openapi.Info(
-            title="Jumpserver API Docs",
-            default_version=version,
-            description="Jumpserver Restful api docs",
-            terms_of_service="https://www.jumpserver.org",
-            contact=openapi.Contact(email="support@fit2cloud.com"),
-            license=openapi.License(name="GPLv2 License"),
-        ),
+        api_info,
         public=True,
         patterns=patterns,
         permission_classes=(permissions.AllowAny,),

apps/ops/ansible/runner.py

@@ -55,7 +55,7 @@ def get_default_options():
     return options
 
 
-# Jumpserver not use playbook
+# JumpServer not use playbook
 class PlayBookRunner:
     """
     用于执行AnsiblePlaybook的接口.简化Playbook对象的使用.