Hi there 👋
About me
🔭 I’m currently working on ColossalAI🌱 HPC-AI Engineer at HPCAI Tech
ver217
Follow
Highlights
- Pro
Pinned
1,452 contributions in the last year
Activity overview
Contributed to
hpcaitech/ColossalAI,
hpcaitech/EnergonAI,
hpcaitech/TensorNVMe
and 21 other
repositories
Contribution activity
March 2023
Created 19 commits in 3 repositories
Created a pull request in hpcaitech/ColossalAI that received 7 comments
[lazyinit] refactor lazy tensor and lazy init ctx
[doc/gemini/tenso…
+170
−214
•
7
comments
Opened 19 other pull requests in 3 repositories
hpcaitech/ColossalAI
1
open
15
merged
- [gemini] gemini supports lazy init
- [booster] implement Gemini plugin
- [coati] fix inference profanity check
- [coati] inference supports profanity check
- [coati] add repetition_penalty for inference
- [coati] fix inference output
- [chatgpt] add precision option for colossalai
- [lazyinit] combine lazy tensor with dtensor
- [chatgpt] fix trainer generate kwargs
- [chatgpt] fix ppo training hanging problem with gemini
- [lazyinit] add correctness verification
- [tests] model zoo add torchaudio models
- [doc] update nvme offload doc
- [doc] add deepspeed citation and copyright
- [chatgpt] allow shard init and display warning
- [chatgpt] making experience support dp
hpcaitech/EnergonAI
2
merged
hpcaitech/FastFold
1
merged
Reviewed 42 pull requests in 3 repositories
hpcaitech/ColossalAI
25 pull requests
- [booster] implement Gemini plugin
- [NFC] polish code
- [doc] add ColossalChat
- [chat]Update Readme
- [booster] implemented the torch ddd + resnet example
- [hotfix] skip torchaudio tracing test
- support instrcut training
- [chatgpt] add pre-trained model RoBERTa for RLHF stage 2 & 3
- [chatgpt] support instuct training
- [api] implemented the checkpoint io module
- [lazyinit] combine lazy tensor with dtensor
- [booster] added the plugin base and torch ddp plugin
- [hotfix] layout converting issue
- [hotfix] Improved flash attention usage
- [booster] added the accelerator implementation
- [test] fixed torchrec model test
- [booster] implemented mixed precision class
- [test] added transformers models to test model zoo
- [test] added torchvision models to test model zoo
- [test] added timm models to test model zoo
- [lazyinit] refactor lazy tensor and lazy init ctx
- [doc] add Intel cooperation for biomedicine
- [kernel] added kernel loader to softmax autograd function
- [chatgpt]update ci
- [doc] moved doc test command to bottom
- Some pull request reviews not shown.
hpcaitech/EnergonAI
2 pull requests
hpcaitech/ColossalAI-Documentation
1 pull request
Created an issue in hpcaitech/ColossalAI that received 2 comments
[lazyinit] add correctness verification
Overview Current implementation is not tested on many models. We have to add large scale correctness verification. Wanna track the development prog…
2
comments
Opened 7 other issues in 1 repository
hpcaitech/ColossalAI
7
closed
- [booster] implement plugin for Gemini
- [BUG]: chatgpt ppo training hangs when using gemini
- [lazyinit] add verification for distributed cases
- [lazyinit] combine lazy tensor with dtensor
- [lazyinit] Implement a single-process version of lazy init
- [PROPOSAL]: refactor core API of Engine
- [FEATURE]: support DP for chatgpt making experience
Started 1 discussion in 1 repository
hpcaitech/ColossalAI
hpcaitech/ColossalAI
-
[PROPOSAL] Lazy initialization of model
This contribution was made on Mar 13
39
contributions
in private repositories
Mar 24 – Mar 28