etl

Tell us about the new connector you’d like to have

• Which source and which destination?
  Source: mambu (https://api.mambu.com/)
  Destination: S3, RDS, Redshift
• Do you need a specific version of the underlying data source e.g: you specifically need support for an older version of the API or DB?
  Want to support for since 1.0.

Describe the context around this new connector

• Why do yo

https://dagster.slack.com/archives/C01U954MEER/p1642163479382400:

Hi, I'm experiencing a bug when trying to write an ALS model from pyspark.ml.recommendation to S3 and reading it back in if this takes place within a dynamically executed graph (i.e. via dynamic mapping). I wrote a custom IO manager using the pattern f's3a://{self.s3_bucket}/{key}' as _uri_for_key similar to the one currently imp

Under the hood, Benthos csv input uses the standard encoding/csv packages's csv.Reader struct.

The current implementation of csv input doesn't allow setting the LazyQuotes field.

We have a use case where we need to set the LazyQuotes field in order to make things work correctly.

https://devblogs.microsoft.com/nuget/introducing-the-new-nuget-org-package-details-page/

https://docs.microsoft.com/en-us/nuget/reference/msbuild-targets#packagereadmefile

The functions in this file should be factored out to a separate utility lib as they are reused in bitcoin-etl https://github.com/blockchain-etl/ethereum-etl/blob/develop/ethereumetl/misc_utils.py

if they are not class methods then the method would be invoked for every test and a session would be created for each of those tests.

`class PySparkTest(unittest.TestCase):
@classmethod
def suppress_py4j_logging(cls):
logger = logging.getLogger('py4j')
logger.setLevel(logging.WARN)

@classmethod
def create_testing_pyspark_session(cls):
    return Sp

Sending a rest call to delete a job specification throws 404 where as grpc call works fine. Steps to reproduce

curl -X DELETE "http://localhost:9100/v1/project/my-project/namespace/kush/helloworld" -H  "accept: application/json"

Right now, these aren't caught until we try to gob-encode. Consider failing faster in type-checking to avoid too much confusion/loss when it works with local execution.

If i have an input yaml connecting to jdbc source like:

inputs:
  somedb:
    jdbc:
      connectionUrl: jdbc:jtds:redact
      user: someuser
      password: somepass

the log has

2020-12-08 17:01:28,076 [main] INFO  com.yotpo.metorikku.Job - these are the config inputs: Some(Map(.....somepass

ideally this should be printed as ****** in the log