建立一個支援LDAP登入+Python2/3的JupyterHub

這幾天被要求弄一個Jupyter給部門的同事用

但是讓人煩惱的是Jupyter預設似乎沒那麼容易支援支援多用戶

後來發現有一個JupyterHub可以用
https://github.com/jupyterhub/jupyterhub

這邊紀錄一下建置方法

先安裝Anaconda2跟Anaconda3來支援python 2/3 以python3為主

$ cd /usr/local
$ wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh
$ sh Anaconda3-5.0.1-Linux-x86_64.sh
...
Anaconda3 will now be installed into this location:
/root/anaconda3

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify a different location below

[/root/anaconda3] >>> /usr/local/anaconda3
..
to PATH in your /root/.bashrc ? [yes|no]
[no] >>> yes


$ wget https://repo.continuum.io/archive/Anaconda2-5.0.1-Linux-x86_64.sh
$ sh Anaconda2-5.0.1-Linux-x86_64.sh
Anaconda3 will now be installed into this location:
/root/anaconda3

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify a different location below

[/root/anaconda3] >>> /usr/local/anaconda2
..
to PATH in your /root/.bashrc ? [yes|no]
[no] >>> no


# check version
$ source ~/.bashrc
$ python -V
Python 3.6.3 :: Anaconda, Inc.

安裝python2 kernel到python3 下

/usr/local/anaconda2/bin/python -m pip install ipykernel
/usr/local/anaconda2/bin/python -m ipykernel install --prefix=/usr/local/anaconda3 --name 'python2'

安裝jupyterhub

$ conda install -c conda-forge jupyterhub
$ conda install notebook
$ pip install jupyterhub-ldapauthenticator
$ jupyterhub --generate-config
$ mkdir -p /etc/juypterhub
$ mv jupyterhub_config.py /etc/juypterhub

# Edit config
$ vim /etc/juypterhub/jupyterhub_config.py
...
c.JupyterHub.ip = '0.0.0.0'
c.JupyterHub.hub_ip = '0.0.0.0'
c.JupyterHub.hub_port = 8081
c.JupyterHub.port = 80
c.JupyterHub.authenticator_class = 'ldapauthenticator.ldapauthenticator.LDAPLocalAuthenticator'
c.LDAPAuthenticator.bind_dn_template = 'uid={username},ou=people,ou=account,ou=development,o=example'
c.LDAPAuthenticator.server_address = 'ldap://myldap.com'
c.LocalAuthenticator.create_system_users = True
c.LocalAuthenticator.add_user_cmd = ['useradd', '-m']
c.LDAPAuthenticator.use_ssl = False

這邊要注意一點,我使用了LDAPLocalAuthenticator當作驗證類別
因為我希望LDAP帳號可以自動在local建立對應的home dir,不然的會出現如下錯誤

Traceback (most recent call last):
  File "/usr/local/anaconda3/lib/python3.6/site-packages/tornado/web.py", line 1511, in _execute
    result = yield result
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/handlers/login.py", line 94, in post
    yield self.spawn_single_user(user)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 475, in spawn_single_user
    yield gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 445, in finish_user_spawn
    yield spawn_future
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/user.py", line 439, in spawn
    raise e
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/user.py", line 378, in spawn
    ip_port = yield gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
  File "/usr/local/anaconda3/lib/python3.6/types.py", line 248, in wrapped
    coro = func(*args, **kwargs)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/spawner.py", line 968, in start
    env = self.get_env()
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/spawner.py", line 960, in get_env
    env = self.user_env(env)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/spawner.py", line 947, in user_env
    home = pwd.getpwnam(self.user.name).pw_dir

而LDAPLocalAuthenticator目前沒有實作在官方的ldapauthenticator裡面
必須自己上patch 請參考: https://github.com/jupyterhub/ldapauthenticator/pull/36/files

$ wget https://patch-diff.githubusercontent.com/raw/jupyterhub/ldapauthenticator/pull/36.patch
$ patch -u /usr/local/anaconda3/lib/python3.6/site-packages/ldapauthenticator/ldapauthenticator.py < 36.patch

之後可以用如下command啟動

#前端模式
$ jupyterhub --no-ssl --config=/etc/juypterhub/jupyterhub_config.py
#背景模式, log導到syslog
$ jupyterhub --no-ssl --config=/etc/juypterhub/jupyterhub_config.py  | logger -t jupyterhub &

備註1:

我在測試途中有踩到一個ldap驗證的問題

  File "/usr/local/anaconda3/lib/python3.6/site-packages/tornado/web.py", line 1511, in _execute
    result = yield result
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/handlers/login.py", line 83, in post
    user = yield self.login_user(data)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/handlers/base.py", line 328, in login_user
    authenticated = yield self.authenticate(data)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/jupyterhub/auth.py", line 227, in get_authenticated_user
    authenticated = yield self.authenticate(handler, data)
  File "/usr/local/anaconda3/lib/python3.6/types.py", line 248, in wrapped
    coro = func(*args, **kwargs)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/ldapauthenticator/ldapauthenticator.py", line 109, in authenticate
    if conn.bind():
  File "/usr/local/anaconda3/lib/python3.6/site-packages/ldap3/core/connection.py", line 594, in bind
    self.refresh_server_info()
  File "/usr/local/anaconda3/lib/python3.6/site-packages/ldap3/core/connection.py", line 1315, in refresh_server_info
    self.server.get_info_from_server(self)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/ldap3/core/server.py", line 446, in get_info_from_server
    self._get_schema_info(connection)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/ldap3/core/server.py", line 431, in _get_schema_info
    self._schema_info.other[attribute] = format_attribute_values(self._schema_info, attribute, self._schema_info.raw[attribute], self.custom_formatter)
  File "/usr/local/anaconda3/lib/python3.6/site-packages/ldap3/protocol/formatters/standard.py", line 200, in format_attribute_values
    formatted_values = [formatter(raw_value) for raw_value in values]  # executes formatter
TypeError: 'NoneType' object is not iterable

似乎是因為ldap3 2.4.0這個版本沒有處理到空的attribute,而理論上LDAP協議裡面也不該出現空的attribute
這應該是作為LDAP Server的實作有不完整的地方
當前dev的版本已經修掉了這個issue且應該會修正到2.4.1,在新版本出來之前,可以先用下面方法對應

$ pip uninstall ldap3
$ git clone -b dev https://github.com/cannatag/ldap3.git
$ cd ldap3
$ python setup.py install

備註2:

如果想用nginx當proxy的話,記得要注意base_url的對應
假設nginx設定/jupyterhub為代理路徑

    location /jupyterhub {
      proxy_pass http://jupyterhub_url:80;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
      proxy_set_header Origin "";
      client_max_body_size 5000m;
      proxy_connect_timeout 1d;
      proxy_send_timeout 1d;
      proxy_read_timeout 1d;
    }

base_url則要跟著設定成/jupyterhub

jupyterhub --no-ssl --config=/etc/juypterhub/jupyterhub_config.py --base-url=/jupyterhub
comments powered by Disqus