正在查看旧版本。 查看 当前版本.

与当前比较 查看页面历史记录

« 前一个 版本 3 当前 »

简介

之前配置了 Seatable- 4.3.8 版本的环境，具体的配置操作参照：https://wiki.waringid.me/x/HQA1AQ以下的操作基于 Seatable-4.4.9。主要实现以下功能：

  • 从 4.3.8 版本升级到 4.4.9 ，确保升级后的数据正常
  • 升级到5.0版本也正常，需要重新调整 seatable.sh 脚本文件。4.4.9版本后重新整合了 api 接口，在启动脚本中增加了 api gateway 的启动，启动端口 7780。
  • 在新版本中配置 python 脚本环境，python 环境注意变量和配置文件的匹配情况
  • 通过 python 脚本实现表格文件自动添加条码和二维码

参考

SeaTable 的 Python 脚本运行包含多个部分，SeaTable, Python scheduler, Python starter 和 Python runner，它们的功能与关系如下

  • SeaTable: 新建/保存/修改脚本，发起运行请求等
  • Python scheduler: 调度器，主要负责调度 SeaTable 运行脚本请求、保存/统计脚本运行结果等。相当于一个 master 节点
  • Python starter: 真正运行脚本，相当于一个 worker 节点。Python starter 在收到一个脚本运行请求后，会下载脚本内容并启动一个 Docker 容器来运行这个脚本。脚本运行结束后，容器自动销毁，以此保证安全性。
  • Python runner: 运行脚本的 Docker 容器。

容器和环境配置

配置 .env 环境脚本

COMPOSE_FILE='seatable-server.yml'
COMPOSE_PATH_SEPARATOR=','
# system settings
TIME_ZONE='Asia/Shanghai'

# seatable server url
SEATABLE_SERVER_HOSTNAME='seatable.waringid.me'
SEATABLE_SERVER_PROTOCOL='https'

# initial web admin
SEATABLE_ADMIN_EMAIL='1377777@139.com'
SEATABLE_ADMIN_PASSWORD='XXXXXX'

# database
SEATABLE_MYSQL_ROOT_PASSWORD='XXXXX'
SEATABLE_DIR='/data/seatable'
SEATABLE_PYTHON_PIPELINE_DB_NAME=scheduler
SEATABLE_PYTHON_PIPELINE_MYSQL_ROOT_PASSWORD='XXXXXX'
SEATABLE_PYTHON_PIPELINE_MYSQL_USER=python
SEATABLE_PYTHON_PIPELINE_MYSQL_PASSWORD='XXXXXXX'

PYTHON_TRANSFER_DIRECTORY_PATH=/data/seatable/python-pipeline/tmp
PYTHON_SCHEDULER_AUTH_TOKEN=25d46ca953db0899c2ca9
PYTHON_SCHEDULER_URL=http://python-scheduler
PYTHON_SCHEDULER_LOGS_DIR=/data/seatable/python-pipeline/logs/scheduler-logs
PYTHON_STARTER_USE_ALTERNATIVE_FILE_SERVER_ROOT=True
PYTHON_STARTER_ALTERNATIVE_FILE_SERVER_ROOT=http://seatable-server
PYTHON_STARTER_LOGS_DIR=/data/seatable/python-pipeline/logs/starter-logs
SEATABLE_SCHEDULER_IMAGE=seatable/seatable-python-scheduler:latest
SEATABLE_STARTER_IMAGE=seatable/seatable-python-starter:3.1.1.p
SEATABLE_RUNNER_IMAGE=seatable/seatable-python-runner:latest
  • PYTHON_STARTER_ALTERNATIVE_FILE_SERVER_ROOT=http://seatable-server  要确保容器内部能正常获取保存的脚本，这里配置内网访问地址，需要同时注意和 nginx 配置文件中的主机头匹配。
  • SEATABLE_STARTER_IMAGE=seatable/seatable-python-starter:3.1.1.p 这个是调整后的镜像文件，原有的镜像配置文件会导致 uwsgi 不断 crash 
  • PYTHON_SCHEDULER_AUTH_TOKEN 为随机字符串（token 可用命令生成 uuidgen） 
  • PYTHON_SCHEDULER_URL 默认为 http://python-scheduler，其中 python-scheduler 是 docker 环境下 SeaTable 访问 Python scheduler 网络名称，不需要修改这个值。
  • Python 运行器共用 SeaTable MySql数据库，并自动读取已有的配置项（SEATABLE_MYSQL_DB_HOST, SEATABLE_MYSQL_ROOT_PASSWORD）
  • 时区也会读取 SeaTable 已有的配置项（TIME_ZONE）
  • 如果不设置 PYTHON_SCHEDULER_LOGS_DIR 或 PYTHON_STARTER_LOGS_DIR 将会默认使用宿主机的 /tmp 目录

修改 seatable 配置文件 conf/dtable_web_settings.py

SEATABLE_FAAS_URL = 'http://python-scheduler'
SEATABLE_FAAS_AUTH_TOKEN = '***'  # 与 .env 文件 PYTHON_SCHEDULER_AUTH_TOKEN 的配置相同

seatable-server.yml

---
services:
  seatable-server:
    image: ${SEATABLE_IMAGE:-seatable/seatable-enterprise:latest}
    restart: unless-stopped
    container_name: seatable-server
    ports:
      - "8006:80"
    volumes:
      - "${SEATABLE_DIR}/seatable-server:/shared"
      - "${SEATABLE_DIR}/deps/seatable.sh:/templates/seatable.sh:rx"
      - "${SEATABLE_DIR}/deps/licenseparse.py:/opt/seatable/seatable-server-latest/dtable-web/seahub/utils/licenseparse.py:rw"
      - "${SEATABLE_DIR}/deps/seafile-controller:/opt/seatable/seatable-server-latest/seafile/bin/seafile-controller:rw"
      - "${SEATABLE_DIR}/deps/seaf-server:/opt/seatable/seatable-server-latest/seafile/bin/seaf-server:rw"
    environment:
      - DB_HOST=mariadb
      - DB_ROOT_PASSWD=${SEATABLE_MYSQL_ROOT_PASSWORD:?Variable is not set or empty}
      - SEATABLE_SERVER_HOSTNAME=${SEATABLE_SERVER_HOSTNAME:?Variable is not set or empty}
      - SEATABLE_SERVER_PROTOCOL=${SEATABLE_SERVER_PROTOCOL:-https}
      - SEATABLE_ADMIN_EMAIL=${SEATABLE_ADMIN_EMAIL:?Variable is not set or empty}
      - SEATABLE_ADMIN_PASSWORD=${SEATABLE_ADMIN_PASSWORD:?Variable is not set or empty}
      - TIME_ZONE=${TIME_ZONE}
    depends_on:
      mariadb:
        condition: service_healthy
      memcached:
        condition: service_healthy
      redis:
        condition: service_healthy
    networks:
      - frontend-net
      - backend-seatable-net
    healthcheck:
      test: ["CMD-SHELL", "curl --fail http://localhost:8000 || exit 1"]
      interval: 20s
      retries: 3
      start_period: 30s
      timeout: 10s

  mariadb:
    image: ${SEATABLE_DB_IMAGE:-mariadb:10.11.6}
    restart: unless-stopped
    container_name: mariadb
    environment:
      - MYSQL_ROOT_PASSWORD=${SEATABLE_MYSQL_ROOT_PASSWORD:?Variable is not set or empty}
      - MYSQL_LOG_CONSOLE=true
      - MARIADB_AUTO_UPGRADE=1
    volumes:
      - "${SEATABLE_DIR}/mariadb:/var/lib/mysql"
    networks:
      - backend-seatable-net
    healthcheck:
      test:
        [
          "CMD",
          "/usr/local/bin/healthcheck.sh",
          "--connect",
          "--mariadbupgrade",
          "--innodb_initialized",
        ]
      interval: 20s
      retries: 3
      start_period: 30s
      timeout: 10s
  memcached:
    image: ${SEATABLE_MEMCACHED_IMAGE:-memcached:1.6.22}
    restart: unless-stopped
    container_name: memcached
    entrypoint: memcached -m 256
    networks:
      - backend-seatable-net
    healthcheck:
      test: ["CMD-SHELL", "timeout 2 bash -c '</dev/tcp/localhost/11211'"]
      interval: 20s
      retries: 3
      timeout: 5s

  redis:
    image: ${SEATABLE_REDIS_IMAGE:-redis:7.2.3}
    restart: unless-stopped
    container_name: redis
    networks:
      - backend-seatable-net
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 20s
      retries: 3
      timeout: 5s

  python-scheduler:
    image: ${SEATABLE_SCHEDULER_IMAGE:-seatable/seatable-python-scheduler:latest}
    restart: unless-stopped
    container_name: python-scheduler
    environment:
      - TIME_ZONE=${TIME_ZONE}
      - DATABASE_NAME=${SEATABLE_PYTHON_PIPELINE_MYSQL_DB_NAME:-scheduler}
      - DB_HOST=mariadb
      - DB_ROOT_PASSWD=${SEATABLE_PYTHON_PIPELINE_MYSQL_ROOT_PASSWORD:-}
      - DB_USER=${SEATABLE_PYTHON_PIPELINE_MYSQL_USER:-}
      - DB_PASSWD=${SEATABLE_PYTHON_PIPELINE_MYSQL_PASSWORD:-}
      - PYTHON_SCHEDULER_AUTH_TOKEN=${PYTHON_SCHEDULER_AUTH_TOKEN:?Variable is not set or empty}
      - SEATABLE_SERVER_URL=${SEATABLE_SERVER_PROTOCOL}://${SEATABLE_SERVER_HOSTNAME}
      - PYTHON_STARTER_URL=${PYTHON_STARTER_URL:-http://python-starter:8080}
      - PYTHON_SCHEDULER_LOG_LEVEL=${PYTHON_SCHEDULER_LOG_LEVEL:-WARNING}
      - DELETE_LOG_DAYS=${DELETE_LOG_DAYS:-30}
      - PYTHON_PROCESS_TIMEOUT=${PYTHON_PROCESS_TIMEOUT:-900}
      - PYTHON_SCHEDULER_SCRIPT_WORKERS=${PYTHON_SCHEDULER_SCRIPT_WORKERS:-5}
    volumes:
      - "${PYTHON_SCHEDULER_LOGS_DIR:-/tmp}:/opt/scheduler/logs"
    extra_hosts:
      - "host.docker.internal:host-gateway"
    networks:
      - backend-scheduler-net
      - frontend-net
      - backend-seatable-net
    depends_on:
      mariadb:
        condition: service_healthy
    healthcheck:
      test:
        [
          "CMD-SHELL",
          "pgrep -f 'python3 scheduler.py' && pgrep -f 'python3 flask_server.py'",
        ]
      interval: 20s
      retries: 3
      start_period: 20s
      timeout: 10s

  python-starter:
    image: ${SEATABLE_STARTER_IMAGE:-seatable/seatable-python-starter:latest}
    restart: unless-stopped
    container_name: python-starter

    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "${PYTHON_TRANSFER_DIRECTORY_PATH:-/tmp}:${PYTHON_TRANSFER_DIRECTORY_PATH:-/tmp}"
      - "${PYTHON_STARTER_LOGS_DIR:-/tmp}:/opt/seatable-python-starter/logs"
    environment:
      - TIME_ZONE=${TIME_ZONE}
      - PYTHON_SCHEDULER_URL=http://python-scheduler
      - PYTHON_TRANSFER_DIRECTORY=${PYTHON_TRANSFER_DIRECTORY_PATH:-/tmp}
      - PYTHON_RUNNER_IMAGE=${SEATABLE_RUNNER_IMAGE:-seatable/seatable-python-runner:latest}
      - PYTHON_STARTER_LOG_LEVEL=${PYTHON_STARTER_LOG_LEVEL:-WARNING}
      - PYTHON_PROCESS_TIMEOUT=${PYTHON_PROCESS_TIMEOUT:-900}
      - PYTHON_STARTER_THREAD_COUNT=${PYTHON_STARTER_THREAD_COUNT:-10}
      - PYTHON_STARTER_USE_ALTERNATIVE_FILE_SERVER_ROOT=${PYTHON_STARTER_USE_ALTERNATIVE_FILE_SERVER_ROOT:-}
      - PYTHON_STARTER_ALTERNATIVE_FILE_SERVER_ROOT=${PYTHON_STARTER_ALTERNATIVE_FILE_SERVER_ROOT:-}
      - PYTHON_RUNNER_OUTPUT_LIMIT=${PYTHON_RUNNER_OUTPUT_LIMIT:-1000000}
      - PYTHON_RUNNER_CONTAINER_MEMORY=${PYTHON_RUNNER_CONTAINER_MEMORY:-2g}
      - PYTHON_RUNNER_CONTAINER_CPUS=${PYTHON_RUNNER_CONTAINER_CPUS:-}
      - PYTHON_RUNNER_OTHER_OPTIONS=${PYTHON_RUNNER_OTHER_OPTIONS:-}
    networks:
      - backend-scheduler-net
      - frontend-net

networks:
  frontend-net:
    name: frontend-net
  backend-seatable-net:
    name: backend-seatable-net
  backend-scheduler-net:
    name: backend-scheduler-net

nginx.conf

  1. 在主机头增加 seatable-server 的 server_name 配置
  2. 在配置文件中增加 api gateway 的配置内容
  3. 本例中的配置文件为 /data/seatable/seatable-server/seatable/conf/nginx.conf
        location /api-gateway/ {
        add_header Access-Control-Allow-Origin *;
        add_header Access-Control-Allow-Methods GET,POST,PUT,DELETE,OPTIONS;
        add_header Access-Control-Allow-Headers "deviceType,token, authorization, content-type";
        if ($request_method = 'OPTIONS') {
            add_header Access-Control-Allow-Origin *;
            add_header Access-Control-Allow-Methods GET,POST,PUT,DELETE,OPTIONS;
            add_header Access-Control-Allow-Headers "deviceType,token, authorization, content-type";
            return 204;
        }
        proxy_pass         http://127.0.0.1:7780/;
        proxy_redirect     off;
        proxy_set_header   Host              $host;
        proxy_set_header   X-Real-IP         $remote_addr;
        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header   X-Forwarded-Host  $server_name;
        proxy_set_header   X-Forwarded-Proto $scheme;
        proxy_hide_header Access-Control-Allow-Origin;
        proxy_hide_header Access-Control-Allow-Methods;
        proxy_hide_header Access-Control-Allow-Headers;
        access_log      /opt/nginx-logs/api-gateway.access.log seatableformat;
        error_log       /opt/nginx-logs/api-gateway.error.log;
    }

seatable/seatable-python-starter:3.1.1.p 镜像

在 Photon5 为宿主机的容器环境下原有的镜像会出现 uwsgi crash 的情况，主要原因是因为配置文件设置存在问题导致

解决的办法是修改配置文件，然后将配置文件打包到镜像文件并在后续的应用中使用新的镜像文件。以下是经测试能正常应用 uwsgi.ini 配置文件内容

 docker  cp ./uwsgi.ini  python-starter:/opt/seatable-python-starter/uwsgi.ini
[uwsgi]
http = :8080
wsgi-file = /opt/seatable-python-starter/runner.py
uid = root
gid = root
chmod-socket = 660
callable = app
max-fd = 1048576
process = 1
threads = 1
vacuum = true
buffer-size = 65536
stats = 127.0.0.1:9191
procname-prefix = run-python
logto = /opt/seatable-python-starter/logs/uwsgi.log
log-level = info

Python 操作及配置

明确 API 和表格内容

  1. 进入系统选择对应的表（Base），选中“Advance”-"API Token"
  2. 在出现的 API 页面 填写对应的描述并选择允许读写 的权限
  3. 将对应的 API 和表格需要操作的项填入脚本文件

打开对应的表，我们需要在“商品档案1”表中按照“商品条码”（字符串类型，数字编码）自动生成对应的“条码”（图片类型，图片文件）

python 脚本文件

import os
import time
import barcode
from barcode.writer import ImageWriter
from seatable_api import Base, context

api_token = context.api_token or "f83c266f818f4df4be30d8fdef8be7b5bbdd5bed"
server_url = context.server_url or "https://seatable.waringid.me"

TEXT_COL = "商品条码"  # column which is expected to be transferred into barcode
BARCODE_IMAGE_COL = "条码"
TABLE_NAME = '商品档案'
BARCODE_TYPE = 'code128'

CUSTOM_OPTIONS = {
    "module_width": 0.2,       # width of single stripe of barcode, mm
    "module_height": 15.0,     # height of barcode, mm
    "quiet_zone": 6.5,         # padding size of first and last stripe to the image, mm
    "font_size": 10,           # font size of the text below the barcode,pt
    "text_distance": 5.0,      # distance between the text and the barcode, mm
}

#print("test",TEXT_COL)
CODE = barcode.get_barcode_class(BARCODE_TYPE)
base = Base(api_token, server_url)
base.auth()

#rows1 = base.list_rows(TABLE_NAME,view_name='default', start=1005, limit=20)
#rows2 = base.get_table_by_name(TABLE_NAME)

def get_time_stamp():
    return str(int(time.time()*100000))

#print("rows1",rows1)
#print("rows2",rows2)
for row in base.list_rows(TABLE_NAME,view_name='default', start=1001, limit=200):
    # continue if the image is already shown up here
    #print("image",row.get(BARCODE_IMAGE_COL))
    if row.get(BARCODE_IMAGE_COL):
        continue
    if not row.get(TEXT_COL):
        continue

    try:
        row_id = row.get('_id')
        msg = str(row.get(TEXT_COL))
        #print("row_id",row_id)
        #print("msg-numer",msg)
        # create a barcode object
        code_img = CODE(msg, writer=ImageWriter())
        save_name = "%s_%s" % (row_id, get_time_stamp())

        # temporarily saved as an image
        file_name = code_img.save("/tmp/%s" % save_name, options=CUSTOM_OPTIONS)

        # upload the barcode image to the base
        info_dict = base.upload_local_file(file_name, name=None, file_type='image', replace=True)
        img_url = info_dict.get('url')
        data = {
          "条码": [img_url]
        }
        #print("img_url",img_url)
        #print("row_id",row_id)
        print("msg-numer-",msg+"ok")
        base.update_row(TABLE_NAME, row_id, data)

        # remove the image file which is saved temporarily
        os.remove(file_name)
    except Exception as error:
        print("error occured during barcode generate", error)
        continue

Notice


  • 无标签