Of the following, c is (by far) the fastest
Dim a = (From X In z.AsParallel
Order By X
Group By X Into Xs = Group
Select X, Xs).ToArray
Dim b = (From X In z.AsParallel
Group By X Into Xs = Group
Order By X
Select X, Xs).ToArray
Dim c = (From X In z.AsParallel
Group By X Into Xs = Group
Select X, Xs).OrderBy(Function(f) f.X).ToArray
that's because it's harder to order the array across multiple threads than to just do the work by thread and sort it on the main thread when completed.