code: https://github.com/FlorinAndrei/c19 (clone this repo)

world.py and usa.py are exported directly from world.ipynb and usa.ipynb. The Jupyter notebooks are the "source of truth".

On macOS and Windows I can run the Jupyter notebooks without any issues.

On Linux (Ubuntu 16.04) I'm trying to run python world.py. It's a headless system, but I've installed Xvfb. I've installed all dependencies via conda, except for fuzzyset which is installed via pip.

Most of world.py works fine on Linux. Then it tries to spawn 4 workers, one for each CPU:

if __name__ == '__main__':
    # one worker per CPU
    workers = psutil.cpu_count(logical = False)
    # MP voodoo. spawn seems to work on Mac and Win.
    # Other methods fail in bizarre ways.
    mp.set_start_method('spawn')

    # ... stuff here ...

    for map_bundle in [(c19g, 'plain numbers'), (c19relg, 'per capita')]:

        # ... stuff here ...

        # Start the pool, then break it down after each data type.
        # Probably safer this way.
        p = Pool(processes = workers)
        work_out = p.map(make_map, warglist)
        p.close()

make_map() is in workers.py. It works on macOS / Win from the Jupyter notebook - it spawns multiple workers, multiple Orca processes. It fails on Linux (see error below). For Linux, I've tried to add plotly.io.orca.ensure_server() - no luck.

Do I need to start one Orca server per worker? On separate ports? Or what's causing the error?

Traceback (most recent call last):
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/connection.py", line 157, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 387, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/home/c19/miniconda3/lib/python3.7/http/client.py", line 1252, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/c19/miniconda3/lib/python3.7/http/client.py", line 1298, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/c19/miniconda3/lib/python3.7/http/client.py", line 1247, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/c19/miniconda3/lib/python3.7/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/c19/miniconda3/lib/python3.7/http/client.py", line 966, in send
    self.connect()
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/connection.py", line 184, in connect
    conn = self._new_conn()
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/connection.py", line 169, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fd77670b690>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/c19/miniconda3/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/c19/miniconda3/lib/python3.7/site-packages/urllib3/util/retry.py", line 436, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=36915): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd77670b690>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/c19/miniconda3/lib/python3.7/site-packages/plotly/io/_orca.py", line 1562, in to_image
    figure=fig_dict, format=format, scale=scale, width=width, height=height
  File "/home/c19/miniconda3/lib/python3.7/site-packages/retrying.py", line 49, in wrapped_f
    return Retrying(*dargs, **dkw).call(f, *args, **kw)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/retrying.py", line 212, in call
    raise attempt.get()
  File "/home/c19/miniconda3/lib/python3.7/site-packages/retrying.py", line 247, in get
    six.reraise(self.value[0], self.value[1], self.value[2])
  File "/home/c19/miniconda3/lib/python3.7/site-packages/six.py", line 703, in reraise
    raise value
  File "/home/c19/miniconda3/lib/python3.7/site-packages/retrying.py", line 200, in call
    attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/plotly/io/_orca.py", line 1471, in request_image_with_retrying
    response = post(server_url + "/", data=json_str)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=36915): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fd77670b690>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/c19/miniconda3/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/c19/miniconda3/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home/c19/git/c19/workers.py", line 211, in make_map
    fig.write_image(map_full_path, scale=1.0)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/plotly/basedatatypes.py", line 3193, in write_image
    return pio.write_image(self, *args, **kwargs)
  File "/home/c19/miniconda3/lib/python3.7/site-packages/plotly/io/_orca.py", line 1770, in write_image
    fig, format=format, scale=scale, width=width, height=height, validate=validate
  File "/home/c19/miniconda3/lib/python3.7/site-packages/plotly/io/_orca.py", line 1594, in to_image
    info=status_str
ValueError:
For some reason plotly.py was unable to communicate with the
local orca server process, even though the server process seems to be running.

Please review the process and connection information below:

orca status
-----------
    state: running
    executable: /usr/bin/xvfb-run --auto-servernum --server-args -screen 0 640x480x24 +extension RANDR +extension GLX /home/c19/miniconda3/bin/orca
    version: 1.3.1
    port: 36915
    pid: 8509
    command: ['/usr/bin/xvfb-run', '--auto-servernum', '--server-args', '-screen 0 640x480x24 +extension RANDR +extension GLX', '/home/c19/miniconda3/bin/orca', 'serve', '-p', '36915', '--plotly', '/home/c19/miniconda3/lib/python3.7/site-packages/plotly/package_data/plotly.min.js', '--graph-only', '--mathjax', 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js']



"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "world.py", line 362, in <module>
    work_out = p.map(make_map, warglist)
  File "/home/c19/miniconda3/lib/python3.7/multiprocessing/pool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/c19/miniconda3/lib/python3.7/multiprocessing/pool.py", line 657, in get
    raise self._value
ValueError:
For some reason plotly.py was unable to communicate with the
local orca server process, even though the server process seems to be running.

Please review the process and connection information below:

orca status
-----------
    state: running
    executable: /usr/bin/xvfb-run --auto-servernum --server-args -screen 0 640x480x24 +extension RANDR +extension GLX /home/c19/miniconda3/bin/orca
    version: 1.3.1
    port: 36915
    pid: 8509
    command: ['/usr/bin/xvfb-run', '--auto-servernum', '--server-args', '-screen 0 640x480x24 +extension RANDR +extension GLX', '/home/c19/miniconda3/bin/orca', 'serve', '-p', '36915', '--plotly', '/home/c19/miniconda3/lib/python3.7/site-packages/plotly/package_data/plotly.min.js', '--graph-only', '--mathjax', 'https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js']