differentiable-programming

As shown in taichi-dev/taichi#3910, replacing property with simple attributes can speedup python part of taichi a lot.
Lessons learned is that we should avoid using @property when applicable since it's expensive. So let's review the usage of @property in our python codebase and replace them as much as possible.

Here's a list of simple grep in our codebase showing

When installing the S4TF toolchain, it's not always clear whether all components are intact and versions are compatible. It would be helpful to have a quick verification tool that uses the toolchain and reports success.

This is especially useful for installations involving accelerators, so the first two features could be:

• Can invoke the toolchain and import TensorFlow
• Can run on

Ankit Shah and I are trying to use Gen to support a project and would love the addition of a dirichlet distribution

Model summary formatting is rudimentary and could be improved so that tabs align. Potentially use some ideas from swift-benchmark.

Generate code coverage reports for our tests.

Debugging Kotlin∇ code within IntelliJ IDEA can be somewhat cumbersome due to the functional API structure (lots of deeply-nested stack traces and context switching). To facilitate more user-friendly debugging, we should add support for visual debugging by exposing Kaliningraph’s built-in graph visualization capabilities. For example, the use

Right now the user can accidentally pass in Duplicated(1.0, 0.0) to autodiff, which is odd and surprising since duplicated is for the shadow storage loacation and 0.0 is immutable and can't be changed on the Julia semantics level. We should use a separate type e.g. Dual for forward diff.

Instead, we could error in forward and backward.

The bug description

The SoftEntropy error function produces NaN values for higher input values.
I identified problems in three different places:

• Softmax computation https://github.com/GustikS/NeuraLogic/blob/master/Algebra/src/main/java/cz/cvut/fel/ida/algebra/functions/specific/SoftEntropy.java#L75
  Math.exp goes to infinity - should be fixable with exp(input[i] - max(input))