Skip to content

/ FASTER

Fast persistent recoverable log and key-value store + cache, in C# and C++.

aka.ms/faster
MIT License
4.7k stars 413 forks
Star
Notifications
master
Switch branches/tags
122 branches 40 tags
Code

Latest commit

@badrishc @rohankadekodi-msr
badrishc and rohankadekodi-msr [C#] Support remote pub-sub and websockets (#543)
6e13ce3 Aug 28, 2021
[C#] Support remote pub-sub and websockets (#543) 
* [C#] Pub/sub functionality in remote FasterKV (#514)

* Add client side support for SubscribeKV

* basic working impl

* Separate span byte serializer into client and server

* Added support in Subscriptions. TODO: prefix matching for subscriptions, publish deletes to keys (#499)

* Added support in Subscriptions. TODO: prefix matching for subscriptions, publish deletes to keys

* Added prefix matching, but similar to redis and unoptimized

* Added null checking for subscriptions on removeSubscriptions

* Made changes to correct concurrency and type checking

* Optimized and corrected the Start() method and Publish() method in publish call

* Added prefix subscriptions. Added a new client serializer call, need to discuss

* Corrected few nits

* Cleanup of code, and resolving NIE

* Nit fix of null checking

* Fixed small nit in MemoryparamSerializer to return Memory<byte> of correct length, and in publish call corner case

* Fixed small nit in MemoryparamSerializer to return Memory<byte> of correct length, and in publish call corner case (#511)

* Resolving merge conflicts

* Removing the subscriptions belonging to a session on disconnection

* Added unit tests and VarLenClient test

* nit fix

* Made broker associated with FasterServer and not FasterKVProvider

* fixed a bug for large values in pub-sub

* Changed the SubscribeKVBroker to be spawned by the user and passed to provider

* fixed nit

* Merged with recent master code

* Checked fixed len client

* Fixed bug in Write of key in server serializer

* Added null check for broker

Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com>

* Improved byte array comparer, moved to FASTER.server

* refactor tests

* [C#] Remote pubsub (#531)

* Add client side support for SubscribeKV

* basic working impl

* Separate span byte serializer into client and server

* Added support in Subscriptions. TODO: prefix matching for subscriptions, publish deletes to keys (#499)

* Added support in Subscriptions. TODO: prefix matching for subscriptions, publish deletes to keys

* Added prefix matching, but similar to redis and unoptimized

* Added null checking for subscriptions on removeSubscriptions

* Made changes to correct concurrency and type checking

* Optimized and corrected the Start() method and Publish() method in publish call

* Added prefix subscriptions. Added a new client serializer call, need to discuss

* Corrected few nits

* Cleanup of code, and resolving NIE

* Nit fix of null checking

* Fixed small nit in MemoryparamSerializer to return Memory<byte> of correct length, and in publish call corner case

* Fixed small nit in MemoryparamSerializer to return Memory<byte> of correct length, and in publish call corner case (#511)

* Resolving merge conflicts

* Removing the subscriptions belonging to a session on disconnection

* Added unit tests and VarLenClient test

* nit fix

* Made broker associated with FasterServer and not FasterKVProvider

* fixed a bug for large values in pub-sub

* Changed the SubscribeKVBroker to be spawned by the user and passed to provider

* fixed nit

* Merged with recent master code

* Checked fixed len client

* Fixed bug in Write of key in server serializer

* Working with non-kv pub-sub. Based off remote-subkv PR

* Fixed some nits, making tests work

* Changed the subscriptions and prefixSubscriptions dictionary to use sid as key

Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com>

* [C#] Remote subkv fixes (#542)

* Add client side support for SubscribeKV

* basic working impl

* Separate span byte serializer into client and server

* Added support in Subscriptions. TODO: prefix matching for subscriptions, publish deletes to keys (#499)

* Added support in Subscriptions. TODO: prefix matching for subscriptions, publish deletes to keys

* Added prefix matching, but similar to redis and unoptimized

* Added null checking for subscriptions on removeSubscriptions

* Made changes to correct concurrency and type checking

* Optimized and corrected the Start() method and Publish() method in publish call

* Added prefix subscriptions. Added a new client serializer call, need to discuss

* Corrected few nits

* Cleanup of code, and resolving NIE

* Nit fix of null checking

* Fixed small nit in MemoryparamSerializer to return Memory<byte> of correct length, and in publish call corner case

* Fixed small nit in MemoryparamSerializer to return Memory<byte> of correct length, and in publish call corner case (#511)

* Resolving merge conflicts

* Removing the subscriptions belonging to a session on disconnection

* Added unit tests and VarLenClient test

* nit fix

* Made broker associated with FasterServer and not FasterKVProvider

* fixed a bug for large values in pub-sub

* Changed the SubscribeKVBroker to be spawned by the user and passed to provider

* fixed nit

* Merged with recent master code

* Checked fixed len client

* Fixed bug in Write of key in server serializer

* Added null check for broker

* Fixed a nit: subscriptions dictionary should contain key as sid and val as serversession

* fixed nit

* Removed old ByteArrayComparer

Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com>

* [C#] Remote websocket (#530)

* Add client side support for SubscribeKV

* basic working impl

* Separate span byte serializer into client and server

* Added support in Subscriptions. TODO: prefix matching for subscriptions, publish deletes to keys (#499)

* Added support in Subscriptions. TODO: prefix matching for subscriptions, publish deletes to keys

* Added prefix matching, but similar to redis and unoptimized

* Added null checking for subscriptions on removeSubscriptions

* Made changes to correct concurrency and type checking

* Optimized and corrected the Start() method and Publish() method in publish call

* Added prefix subscriptions. Added a new client serializer call, need to discuss

* Corrected few nits

* Cleanup of code, and resolving NIE

* Nit fix of null checking

* Fixed small nit in MemoryparamSerializer to return Memory<byte> of correct length, and in publish call corner case

* Fixed small nit in MemoryparamSerializer to return Memory<byte> of correct length, and in publish call corner case (#511)

* Resolving merge conflicts

* Removing the subscriptions belonging to a session on disconnection

* Added unit tests and VarLenClient test

* nit fix

* Made broker associated with FasterServer and not FasterKVProvider

* fixed a bug for large values in pub-sub

* Changed the SubscribeKVBroker to be spawned by the user and passed to provider

* fixed nit

* Merged with recent master code

* Checked fixed len client

* Working on JS client

* Added batching in websocket handling. Next step: finish HandlePending()

* Modified websocket server. Fixed bug in Write key of server serializer

* duplicate file from faster.common

* Added sample for WebClient

* fixed small nit

* Fixed some nits in js clientsession

* fixed nit

* removed old ByteArrayComparer

* Fixed nits, added non-kv PubSub to WebsocketServerSession

Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com>

* Cleanup and updates.

* fix break.

* [C#] Modifications to pubsub (#547)

* Added FASTER Log for SubscribeKV and Subscribe

* Added input in subscribeKV

* Added SubscribeCallback and SubscribeKV sample in html

* set enablePubSub: false for non-pubsub tests

Co-authored-by: rohankadekodi-msr <69916400+rohankadekodi-msr@users.noreply.github.com>
6e13ce3

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
cc
[C++] Rmw bugfix (#533)
Jul 28, 2021
cs
[C#] Support remote pub-sub and websockets (#543)
Aug 28, 2021
docs
Update 80-build-and-test.md
Aug 19, 2021
img
Add files via upload
Aug 27, 2020
.gitattributes
Initial commit
Aug 14, 2018
.gitignore
[C#] Add infrastructure to support more wire protocols (#496)
Jun 18, 2021
CONTRIBUTING.md
Create CONTRIBUTING.md
Jan 14, 2019
LICENSE
Added file LICENSE
Aug 14, 2018
README.md
Add a link to the DPR branch (#506)
Jun 22, 2021
azure-pipelines-full.yml
[C#] Multiple device tests (#523)
Jul 20, 2021
azure-pipelines.yml
[C#] Multiple device tests (#523)
Jul 20, 2021
Introduction Contributing

README.md

FASTER logo

NuGet Build Status Gitter

Introduction

Managing large application state easily, resiliently, and with high performance is one of the hardest problems in the cloud today. The FASTER project offers two artifacts to help tackle this problem.

  • FASTER Log is a high-performance concurrent persistent recoverable log, iterator, and random reader library in C#. It supports very frequent commit operations at low latency, and can quickly saturate disk bandwidth. It supports both sync and async interfaces, handles disk errors, and supports checksums.

  • FASTER KV is a concurrent key-value store + cache (available in C# and C++) that is designed for point lookups and heavy updates. FASTER supports data larger than memory, by leveraging fast external storage (local or cloud). It also supports consistent recovery using a fast non-blocking checkpointing technique that lets applications trade-off performance for commit latency.

Both FASTER KV and FASTER Log offer orders-of-magnitude higher performance than comparable solutions, on standard workloads. Start learning about FASTER, its unique capabilities, and how to get started at our official website:

aka.ms/FASTER

For a preview of our new distributed recovery algorithm of DPR, please take a look at the DPR development branch

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

About

Fast persistent recoverable log and key-value store + cache, in C# and C++.

aka.ms/faster

Topics

library logging persistent indexing concurrent hash-table key-value-store multi-threaded recoverable

Resources

Readme

License

MIT License

Releases 40

FASTER v1.9.5 Latest
May 29, 2021
+ 39 releases

Packages

No packages published

Contributors 49

+ 38 contributors

Languages