Concurrency is a programming abstraction that conceptually allows multiple independent processes to run at the same time using scheduling. For example, running ten programs at once with only a single CPU. Concurrency is effectively CPU scheduling with maybe some other concepts of communication between processes for coordination.
Parallelism is running processes at the same time.
The driving motivation behind concurrency, as far as I can tell, is that it's a concept that can help reason about complex tasks with the added benefit of being able to take advantage of parallelism.
My problem with Pike's advocacy for concurrency, and Golang in particular, is that it's an abstraction that imposes a world view on what's a good programming paradigm and, more practically, how to take advantage of parallelism, both of which have limited utility compared with considering parallelism directly.
My opinion rather than some abstraction that imposes a world view on how to take advantage of parallelism, it's better to use how people actually take advantage of parallelism as a basis for abstraction.
I think Golang is a fine language but we now have 14+ years of hindsight to see how well it's fared and how much it's influenced the compute landscape. In my opinion, the "concurrency not parallelism" misses the simple issue that we want things to go fast. Which has had more influence, Golang's idea of concurrency or taking advantage of GPU parallelism? I think the answer is clearly taking advantage of GPU parallelism.
> For example, running ten programs at once with only a single CPU. Concurrency is effectively CPU scheduling with maybe some other concepts of communication between processes for coordination.
This is called preemptive multitasking. This is like declaring that "sorting a list is quicksort." This can realize concurrency.
The reason that this is important is that cooperative multitasking (async/cps/fibers/safe points) is also concurrency. And I'm sure there are other exotic forms of concurrency that I'm not aware of beyond multitasking.
Concurrency is a property of a program/system, not a mechanism.
Parallelism is like data structures. There are many different ways to do it and people should not be trying to do it themselves in a bespoke way on every program.
Having one way built into the language is like a language using a hash map for everything. It might be convenient when it works, but the idea that there is one best way is only a little better than people fumbling their way through it from scratch.
I agree with a lot of this sentiment but it assumes a level of stability so that standardization can be effective. Standardization before stabilization has the potential to cripple efficiency with inefficient abstractions.
One benefit of a programming paradigm is that it can allow for an increase in speed of software development at a marginal cost of run-time efficiency, but if the paradigm is so out of step with the underlying hardware, the run-time efficiencies can be orders of magnitude slower than a bespoke way.
Standardization and abstraction are most effective when there is data about which bespoke ways work the best. Choosing standardization and abstraction before experimenting on the solution space rarely works well.
I'm not sure what you mean about standardization, just like data structures parallelism techniques can be done through libraries, they don't need to be integrated into a language if the core tools are there.
yeah, i use that often, eg when a video is slow on a website - at least yt-dlp will shard out multiple piece-downloaders, and I can get the video in a few seconds or minutes, and just watch it. Also great for anonymizing sharing videos - download the actual video, share that. Not an url (:
In much the same vein, I rarely actually watch stuff _on_ netflix, through a browser - I watch sped up, and the quality just degrades. Since I pay for it, I feel nothing for downloading a ripped copy to watch it locally :D
it works for me without login or subscription. no idea. but at least i an confirm that the youtube video is the same one.
you also want to open the slides and run them in parallel because the video doesn't show them very well. the slides are linked in the youtube description, but i don't see them linked on vimeo: https://talks.golang.org/2012/waza.slide
After watching the video I can see how go lang makes it easier to write correct concurrent programs. But with AI writing the code these days, it is just as "easy" to write correct concurrent programs in Java (because AI is doing the work). Java's virtual threads are light weight, just like go's routines. Java's LinkedBlockingQueue offers roughly the same functionality as go's channels. I would like to hear from go experts as to why I am wrong. Does go have any inherent advantage if AI is writing the code?
It seems like we forgot about lightweight fibers for about 30 years and then collectively rediscovered it in about 2010. sure, Java did green threads, but not like m:n stuff.
I sometimes wonder where we would be now if people would have gone "Wow, mr Reppy! concurrentML is so cool!" in 1993.
instead we got pthreads and collective amnesia and later we got go's girly times and channels which are only half way there.
That's because they are very rarely useful. This was true then and it's still true now. There's just not many workloads where it makes sense to need to rapidly launch a thread that doesn't need to do much of anything but does need to exist for a while before terminating.
What is useful is the state machine aspects of things like coroutines or async/await, but those aren't quite fibers and very much aren't M:N threading. A major use of them is in UI where they have strict thread requirements even.
Exactly. When I started getting into go 15 years ago, and 10 years ago or so into elixir, I loved the easy concurrency and tried to use it to the maximum possible. What I discovered is that they're just aren't that many things that you can do concurrently.
Reading many different files or servicing many different network sockets being the exception, but realistically that isn't done very often relative to single threaded stuff.
State machine/management is really what is most useful
Back then you could spend 6 months making something twice as fast, or wait six months for computers to become twice as fast. People obviously did the latter.
It also became a cycle: People write single threaded code -> CPUs/OSs optimize performance/ergonomics for single threaded programs -> People write single threaded code ->...
It wasn't until scaling slowed down that interest/investment in concurrency/parallelism took off.
Yet the average software engineer hasn’t heard or Carl Hewitt, Joe Armstrong, has never programmed in an Actor Programming model, and dont even know what it is.
concurrency _is_ parallelism, but for I/O. People often think of parallelism for the case of making something go faster - eg placing two computations in parallel (the definition posed in the video), OR placing two I/O operations in parallel - so this is the keyboard-vs-mouse in the OS, even when you're on one core only; this is multiple web requests in JavaScript, which does not support multi-threading, but 100% does support concurrency for I/O operations - that... badum-tiss! RUN IN PARALLEL.
I get the point of the talk, and it's well interesting, but I think it depends on how one views things.
Not really. They're just separate but related concepts.
E.g. coroutines are a form of concurrency that doesn't have to involve any sort of I/O, you're just taking two logical processes (e.g generating a sequence and consuming it) and abstracting away how they execute relative to each other.
Describing your tasks using the language of concurrency is a requirement for process-based parallelism (multiple CPUs/cores), but data-level parallelism (SIMD) is a form of parallelism that doesn't involve concurrency either.
No, and that's the point of the article. What you are calling parallel w/r/t IO should be called concurrency (conceptually happening at the same time by virtue of being able to interrupt and resume units of work). The reason IO APIs like you've described is concurrent but not necessqrily parallel is because there is no guarantee in the API that they both happen literally simultaneously; I could build a JS runtime that "works" for all the code written against XMLHTTPRequest (ignoring side-effects) but which under the hood only ever makes one HTTP request at a time. And because I can do that, that means JS code is living in a concurrency-only world, even though as an implementation detail most runtimes support parallel execution of those concurrent operations.
> there is no guarantee in the API that they both happen literally simultaneously
There's no actual guarantee in the API that if you spawn multiple threads and call blocking network I/O that those happen literally simultaneously. Maybe the OS has a big mutex on network I/O to serialise them.
Of course, that's not what happens in practice. But neither is it what happens, in practice, to async network APIs called concurrently in one thread. So I don't think that can be the difference between concurrent and parallel.
Concurrency is a programming abstraction that conceptually allows multiple independent processes to run at the same time using scheduling. For example, running ten programs at once with only a single CPU. Concurrency is effectively CPU scheduling with maybe some other concepts of communication between processes for coordination.
Parallelism is running processes at the same time.
The driving motivation behind concurrency, as far as I can tell, is that it's a concept that can help reason about complex tasks with the added benefit of being able to take advantage of parallelism.
My problem with Pike's advocacy for concurrency, and Golang in particular, is that it's an abstraction that imposes a world view on what's a good programming paradigm and, more practically, how to take advantage of parallelism, both of which have limited utility compared with considering parallelism directly.
My opinion rather than some abstraction that imposes a world view on how to take advantage of parallelism, it's better to use how people actually take advantage of parallelism as a basis for abstraction.
I think Golang is a fine language but we now have 14+ years of hindsight to see how well it's fared and how much it's influenced the compute landscape. In my opinion, the "concurrency not parallelism" misses the simple issue that we want things to go fast. Which has had more influence, Golang's idea of concurrency or taking advantage of GPU parallelism? I think the answer is clearly taking advantage of GPU parallelism.
> For example, running ten programs at once with only a single CPU. Concurrency is effectively CPU scheduling with maybe some other concepts of communication between processes for coordination.
This is called preemptive multitasking. This is like declaring that "sorting a list is quicksort." This can realize concurrency.
The reason that this is important is that cooperative multitasking (async/cps/fibers/safe points) is also concurrency. And I'm sure there are other exotic forms of concurrency that I'm not aware of beyond multitasking.
Concurrency is a property of a program/system, not a mechanism.
What languages do you like?
Parallelism is like data structures. There are many different ways to do it and people should not be trying to do it themselves in a bespoke way on every program.
Having one way built into the language is like a language using a hash map for everything. It might be convenient when it works, but the idea that there is one best way is only a little better than people fumbling their way through it from scratch.
I agree with a lot of this sentiment but it assumes a level of stability so that standardization can be effective. Standardization before stabilization has the potential to cripple efficiency with inefficient abstractions.
One benefit of a programming paradigm is that it can allow for an increase in speed of software development at a marginal cost of run-time efficiency, but if the paradigm is so out of step with the underlying hardware, the run-time efficiencies can be orders of magnitude slower than a bespoke way.
Standardization and abstraction are most effective when there is data about which bespoke ways work the best. Choosing standardization and abstraction before experimenting on the solution space rarely works well.
I'm not sure what you mean about standardization, just like data structures parallelism techniques can be done through libraries, they don't need to be integrated into a language if the core tools are there.
Why does Vimeo require me to verify my age in the UK to watch Robert Pike talking computer science?
Surveillance, got to track you all across the internet.
In Greece (EU). Can’t play it either. I’m getting:
> This video is not rated
> Join vimeo to watch
> Already have an account? Log in
through a german proxy it works just fine.
Compliance with the absurd and draconian OSA
Alt link: https://www.youtube.com/watch?v=54EV4dDdLO0
because otherwise how would the queen know what you're up to all the time? :D
Since she was probably sent by God to rule, I assume he called her back upstairs upon her passing. In this case, she can just ask the big man.
Funnily enough, yt-dlp has no trouble downloading it.
Side note: we do have a Queen. She's married to the King.
Hah! Fair point. Not sure why she would want to surveil me, but at this point...
yeah, i use that often, eg when a video is slow on a website - at least yt-dlp will shard out multiple piece-downloaders, and I can get the video in a few seconds or minutes, and just watch it. Also great for anonymizing sharing videos - download the actual video, share that. Not an url (:
In much the same vein, I rarely actually watch stuff _on_ netflix, through a browser - I watch sped up, and the quality just degrades. Since I pay for it, I feel nothing for downloading a ripped copy to watch it locally :D
Slides - https://go.dev/talks/2012/waza.slide
I won’t subscribe to watch a video, sorry. I think this is the YouTube version though: https://youtu.be/oV9rvDllKEg
it works for me without login or subscription. no idea. but at least i an confirm that the youtube video is the same one.
you also want to open the slides and run them in parallel because the video doesn't show them very well. the slides are linked in the youtube description, but i don't see them linked on vimeo: https://talks.golang.org/2012/waza.slide
After watching the video I can see how go lang makes it easier to write correct concurrent programs. But with AI writing the code these days, it is just as "easy" to write correct concurrent programs in Java (because AI is doing the work). Java's virtual threads are light weight, just like go's routines. Java's LinkedBlockingQueue offers roughly the same functionality as go's channels. I would like to hear from go experts as to why I am wrong. Does go have any inherent advantage if AI is writing the code?
It seems like we forgot about lightweight fibers for about 30 years and then collectively rediscovered it in about 2010. sure, Java did green threads, but not like m:n stuff.
I sometimes wonder where we would be now if people would have gone "Wow, mr Reppy! concurrentML is so cool!" in 1993.
instead we got pthreads and collective amnesia and later we got go's girly times and channels which are only half way there.
That's because they are very rarely useful. This was true then and it's still true now. There's just not many workloads where it makes sense to need to rapidly launch a thread that doesn't need to do much of anything but does need to exist for a while before terminating.
What is useful is the state machine aspects of things like coroutines or async/await, but those aren't quite fibers and very much aren't M:N threading. A major use of them is in UI where they have strict thread requirements even.
Exactly. When I started getting into go 15 years ago, and 10 years ago or so into elixir, I loved the easy concurrency and tried to use it to the maximum possible. What I discovered is that they're just aren't that many things that you can do concurrently. Reading many different files or servicing many different network sockets being the exception, but realistically that isn't done very often relative to single threaded stuff.
State machine/management is really what is most useful
Back then you could spend 6 months making something twice as fast, or wait six months for computers to become twice as fast. People obviously did the latter.
It also became a cycle: People write single threaded code -> CPUs/OSs optimize performance/ergonomics for single threaded programs -> People write single threaded code ->...
It wasn't until scaling slowed down that interest/investment in concurrency/parallelism took off.
Yet the average software engineer hasn’t heard or Carl Hewitt, Joe Armstrong, has never programmed in an Actor Programming model, and dont even know what it is.
"girly times" = goroutines?
To HN moderators: title needs to note that this talk is 13 years old.
The title must have been edited in the last 40min - it shows (2012) now
Just set the source pile of C++ manuals on fire. No gophers needed!
it's an interesting talk, my take being:
concurrency _is_ parallelism, but for I/O. People often think of parallelism for the case of making something go faster - eg placing two computations in parallel (the definition posed in the video), OR placing two I/O operations in parallel - so this is the keyboard-vs-mouse in the OS, even when you're on one core only; this is multiple web requests in JavaScript, which does not support multi-threading, but 100% does support concurrency for I/O operations - that... badum-tiss! RUN IN PARALLEL.
I get the point of the talk, and it's well interesting, but I think it depends on how one views things.
> concurrency _is_ parallelism, but for I/O.
Not really. They're just separate but related concepts.
E.g. coroutines are a form of concurrency that doesn't have to involve any sort of I/O, you're just taking two logical processes (e.g generating a sequence and consuming it) and abstracting away how they execute relative to each other.
Describing your tasks using the language of concurrency is a requirement for process-based parallelism (multiple CPUs/cores), but data-level parallelism (SIMD) is a form of parallelism that doesn't involve concurrency either.
Concurrency is the property of a program or algorithm such that:
Your data-level parallelism is taking advantage of the concurrent properties of a problem.No, and that's the point of the article. What you are calling parallel w/r/t IO should be called concurrency (conceptually happening at the same time by virtue of being able to interrupt and resume units of work). The reason IO APIs like you've described is concurrent but not necessqrily parallel is because there is no guarantee in the API that they both happen literally simultaneously; I could build a JS runtime that "works" for all the code written against XMLHTTPRequest (ignoring side-effects) but which under the hood only ever makes one HTTP request at a time. And because I can do that, that means JS code is living in a concurrency-only world, even though as an implementation detail most runtimes support parallel execution of those concurrent operations.
> there is no guarantee in the API that they both happen literally simultaneously
There's no actual guarantee in the API that if you spawn multiple threads and call blocking network I/O that those happen literally simultaneously. Maybe the OS has a big mutex on network I/O to serialise them.
Of course, that's not what happens in practice. But neither is it what happens, in practice, to async network APIs called concurrently in one thread. So I don't think that can be the difference between concurrent and parallel.
concurrent programs enable parallel evaluation. concurrency is necessary but not sufficient for parallelism.