etl

Tell us about the problem you're trying to solve

We can probably reduce the docker image size of our java based connectors by using the ADD command instead of COPYing the tar archive. See this PR for an example

Describe the solution you’d like

use the ADD command to reduce the size of the docker images

Issue from the Dagster Slack

[dagster_shell] defer environ copy to solid run time

This issue was generated from the slack conversation at: https://dagster.slack.com/archives/C01U954MEER/p1623181007055200?thread_ts=1623181007.055200&cid=

In .NET Trim, TrimEnd, TrimStart all can take an optional array of characters that sould be trimmed.

With current 3.x release, we observed that when we use this optional parameter:

1. "abc".Trim('c') throws a LinqException saying it can't be translated.
2. "abc".TrimEnd('c') actually runs without exception, but silently drops the 'c' parameter!!

It would be nice if 2. would

问题描述：关于工程中某个工作流节点单独执行不报错，保存工作流时提示：NotClassDefFoundError：Cloud not initialize class dispatch.Http$

问题原因：linkis-publish微服务关于netty-3.6.2.Final.jar升级包缺失

问题解决：将升级包上传重启linkis-publish微服务即可。

The functions in this file should be factored out to a separate utility lib as they are reused in bitcoin-etl https://github.com/blockchain-etl/ethereum-etl/blob/develop/ethereumetl/misc_utils.py

if they are not class methods then the method would be invoked for every test and a session would be created for each of those tests.

`class PySparkTest(unittest.TestCase):
@classmethod
def suppress_py4j_logging(cls):
logger = logging.getLogger('py4j')
logger.setLevel(logging.WARN)

@classmethod
def create_testing_pyspark_session(cls):
    return Sp

Sending a rest call to delete a job specification throws 404 where as grpc call works fine. Steps to reproduce

curl -X DELETE "http://localhost:9100/v1/project/my-project/namespace/kush/helloworld" -H  "accept: application/json"

Right now, these aren't caught until we try to gob-encode. Consider failing faster in type-checking to avoid too much confusion/loss when it works with local execution.