ParallelStream

Parallelized stream implementation for elixir

What does it do?

Parallelize some stream operations in Elixir whilst keeping your stream in order. Operates with a worker pool.

Add

{:parallel_stream, "~> 1.0.5"}

to your deps in mix.exs like so:

defp deps do
  [
    {:parallel_stream, "~> 1.0.5"}
  ]
end

Note: Elixir 1.1.0 is required

Do this to parallelize a map:

stream = 1..10 |> ParallelStream.map(fn i -> i * 2 end)
stream |> Enum.into([])
[2,4,6,8,10,12,14,16,18,20]

The generated stream is sorted the same as the input stream.

More supported functions are each (to produce side-effects):

1..100 |> ParallelStream.each(&IO.inspect/1)

filter:

stream = 1..20 |> ParallelStream.filter(fn i -> i |> rem(2) == 0 end)
stream |> Enum.into([])
[2,4,6,8,10,12,14,16,18,20]

and filter's counterpart, reject:

stream = 1..20 |> ParallelStream.reject(fn i -> i |> rem(2) == 0 end)
stream |> Enum.into([])
[1,3,5,7,9,11,13,15,17,19]

MIT

Failed to load latest commit information.
bench	Cleaner benchmark	Apr 16, 2016
lib	Use parentheses to remove the ambiguity of zero-arity-function calls	Jan 18, 2017
test	Use parentheses to remove the ambiguity of zero-arity-function calls	Jan 18, 2017
.gitignore	Add benchmark to illustrate #1	Apr 16, 2016
.travis.yml	Update elixir versions and add 1.4.x, 1.5.x	Aug 10, 2017
LICENSE	Initial commit	Oct 17, 2015
README.md	Update README	Jun 22, 2016
mix.exs
mix.lock	Update lockfile	Dec 20, 2017