I've observed an increasing trend with modern AI research that
A) It's no longer practical to incorporate into real products given the hardware costs for training and inference. In NLP, large transformer based models can easily occupy a 5 thousand dollar GPU at a paltry 50-60 words per second processing pace.
B) The Research often demonstrates things that existing non-ML toolchains were already reasonable at doing with a better result, and a faster iteration time.
This demo looks like it's replicated the core mechanics of pac-man with worse graphics, 3-4 orders of magnitude higher resource consumption, and undoubtably many odd bugs hidden away from the pretty demo. It even needed the game to exist in the first place! While this is fascinating from the technical and academic perspective, it's hard to be convinced that any gains we see in the field won't be offset by increased hardware/latency costs from a product standpoint.
> It's no longer practical to incorporate into real products given the hardware costs for training and inference
I don't see that trend. Bleeding edge research has always been beyond what's practical, by its very nature. The focus is on pushing the boundaries, not to build something efficient/commercial.
It's the followup work that makes it practical. See for example the evolution of WaveNet. It was breakthrough research, but yeah it wasn't practical. Now a couple years later and we're generating speech on mobile devices of similar quality (see the recent Facebook research), and able to teach these machines using orders of magnitude less data (see 15.ai).
Not to mention the plethora of research that is explicitly about making neural networks more efficient and easier to train (e.g. EfficientNet; 1cycle; etc).
Bleeding edge stuff like AlphaStar, BigGAN, GPT-2, etc are just that: bleeding edge. They'll get more efficient and practical every year, just like everything else in machine learning.
It reminds me of older electric cars. They worked. They could get you around. But they did so at higher price, slower speeds, or other factors that a consumer wasn't willing to pay for. Thus they were regulated to novelty items that some work campuses would have and concept cars that would be shown from time to time.
But eventually we reached the point where they are not only competitive, they may be positioned to fully overtake combustion cars.
I wonder how many discoveries on electric cars ended up being fully dead ends along the path that took us to the point we are now?
Electric cars of the era you describe could at least go. The approach described in this paper doesn't "go". Its goal is to "learn a simulator [to train a robotic agent] simply by watching an agent interacting with the environment".
The learning part is done by training on the output of an existing simulator. Obviously, if you already have the output of a simulator, you don't need to learn a simulator so the only real use of this approach is to train a simulator by watching a human agent interacting with the environment. So the idea is to eventually apply this thing to training robots to interact in the real world by watching how humans do it (this is not clearly stated in the paper; it's my charitable interpretation of the very poor motivation of learning a simulator from the output of an existing simulator).
The only problem is that creating an accurate simulation of a human's interaction with the real world (let alone an accurate simulation of the real world) is prohibitively expensive. Thus, for the time being, this kind of idea works only in very, very simple environments like PacMan.
So we don't yet have an expensive prototype electic car that nevertheless can do everything an er other kind of car can do. We have... not sure what. The description of an idea with an example of how such a thing could be achieved in the far future, when we can train robots to act like humans by watching us.
It seems like there is not a lot of novel technology in an EV; what made them viable was steady improvements in battery technology so that the intersection between range, price, and weight is in a place that is financially feasible. Combined with continued improvements in battery, the already huge gain in fuel efficiency (for a typical car at least) will be what causes them to overtake ICEVs.
Thanks. It seems obvious in hindsight. I wonder if I had the correct word in my head but my fingers typed the wrong word. That's happened a few times when dealing with words I rarely type. Sadly too late to edit.
Your comment reminds me of snide dismissals of personal computing in the 1980s - looks worse than [existing thing], very expensive, requires an army of technicians, probably explodes unpredictably, overall just a terrible investment.
Followed by decades long monopoly over expensive new technology, likely crushing waves of innovation and usage, in exchange for slow, buggy, not-even-automated spreadsheets; plus the hemming of governments and organizations into endless software rental contracts and platforms, that maybe don't add value as fast as Moore's Law.
that's a fair criticism, however it's not a given that any product category will end up with the same growth curve of PCs, the web, cloud, and mobile.
thinking back on the PC, Web, cloud, and mobile growth periods, each came with a transformative change in the capabilities, cost, and availability of computing power. We're only seeing two of these aspects in AI today - with the cost side heading in the wrong direction.
AI supercharged the importance of computing power to arms-race levels. We're still just brute forcing it with Deep Learning, but the capabilities are too awesome to ignore.
Therefore, compute power will be bought-up and hoarded expensively, and used on the highest bidder. Will the market save us, and simply over-clock production? Not happening yet. Corporate appetite should be quite large. I do A.I. R&D and am just watching the mountain of power ascend before me and aged gaming laptop. My next level is yet the foothills. Yeah, I'd say growth or not, A.I. is going where PC, web, cloud, everything, goes, to the industrial monopolists and the money spenders.
I work in industrial AI research as well, but I think it's worth considering whether AI is providing an incremental benefit on existing products through the use of imense computing power - or unlocking new demand through new applications. If the latter category is limited by available compute, then we'll reach the limits of economically viable use cases rather quickly without the aid of moore's law.
This is obviously not meant to be a real game engine. It shows that GANs can learn not only images but transitions between images given some action, which means they can be used as simulators. See world-models which used VAEs https://worldmodels.github.io/
The english longbow was vastly superior to the first guns that came out, which were slow to load, unreliable to shoot, didn't work in the rain, and were extremely inaccurate.
3d printed stuff sucks. It's not very strong, it's not very pretty and takes a long time to create.
Meanwhile, take a look at this ML photo processing technique:
Do you really think nVidia is seriously advocating for using neural networks as the best way to write a PacMan clone? It's obviously just a proof of concept to demonstrate that GANs are sophisticated enough to learn certain categories of models.
Doing these cute demos is a double-edged sword. A fun and charming demo attracts attention by non-experts who can't see the wood for the trees.
In my field we make robots do a thing, say folding laundry, as an example of dextrous manipulation[1], and the popular conversation is about whether your laundry robot makes commercial sense and not about your novel grasping strategy and nifty motion planner that makes this sorta-possible for the first time. But everyone sees your video. Pros and cons.
A) is just a sign, that further engineering, both model and framework, is required. Also, 50-60 words per second does not sound too bad. It amounts to 4M words per day. I imagine most people only type a thousand words per day. So with a single GPU you can handle 4000 users (assuming batch-style processing).
B) This specific example actually shows ML toolchains are orders of magnitude better, than existing ones. Imagine you wanted to make a football game from scratch, and this technique would let you do that by just feeding a lot of football game recordings to a program, as opposed to hiring software devs and graphics designers, and iterating for a year.
On A: Using a p3.2xl for that task leaves you with a minimum ec2 cost of goods sold of 50 cents per user per month for the target task. To make a reasonable SaaS business out of this you would need to be charging a minimum of $5.00 per user month for the service that this GPU is providing, assuming all other tasks have a negligible impact on Cost of Goods Sold, and that you are able to spread the load optimally throughout the day - and thus don't have to purchase higher capacity during peak.
This isn't too bad of a situation if this GPU provides 100% of your value proposition. but, if you needed your product to perform 5 such tasks - or competition/customer demand forces you to move to a bigger model with a 10 words per second inference rate then you may find it hard to balance customer value vs. price vs. margin.
"This demo looks like it's replicated the core mechanics of pac-man with worse graphics, 3-4 orders of magnitude higher resource consumption, and undoubtably many odd bugs hidden away from the pretty demo. It even needed the game to exist in the first place!"
True, though my first thought was that this would be interesting from a reverse engineering point of view. Then again Pacman is pretty simple concept. Something worth reverse engineering is probably a lot more complex.
Like every so-called basic science. 1st batteries, radio, lasers etc also looked like that.
I see a strong parallel. It's a poor performance but important proof of concept.
Also, it's general in computing trend since assembly. Why compile when you can write machine code, waste of resources. Why have abstract compiler, when you can write assembly. Why have higher level language when it cant generate code optimal for target when it doesnt know it. Then web came, it's not faster than 20 years ago because with increased computing power bloat came. Somehow it always turns that seemingly wasteful apporach outweights its cons. Empirically lowered entry cost and increased applicability has very very high value. At least higher that I used to assess.
There is a reason why "Research" and "Development" are listed as separate concepts when people talk about R&D. The point of most research papers is not to directly invent a working product, it's to produce fundamental insights that may be useful in future applications.
See my previous comment: not in machine learning research.
Also, the paper itself tries (hard) to motivate its approach with a real-world application, in particular, learning a simulator to train robotic agents to interact with the real world.
It really is a great paper, the problem is when your entire field has few real applications but everyone is making too much money to admit that. Cool tech demos like this is awesome as a blog post, but not sure why it's academic...
Well, the paper has an experiment where they train an agent based on their system to play two of the three games (PacMan, PacManMaze and VizDoom) and compare its result to two other systems (an LSTM-based one and a model-based RL one, WorldModel). Their system gets the best score in Pacman but unfortunately the version that does better than WorldModel is the one that doesn't have the memory module whose usefulness is a central claim of their paper. WorldModel outperforms their system in VizDoom by ~300 points, though their system without the memory module "solves" VizDoom (i.e. scores just a bit over than 750; that is 765) and they claim that it's the first Gan-based system that achieves this.
A bit thin of the ground (especially when it comes to supporting the claim about the usefulness of the memory module) but the way neural nets research and especially RL research is going (from what I've seen, not my area of expertise) that's the kind of result that you can expect will get your paper published these days.
I don't know where you got the idea that the purpose of an academic paper is to produce the schematics for a working commercial product. The fact that it's novel and interesting is sufficient grounds for producing a paper about it. Why do you think projects like these are solely the domain of blog posts?
That is not true for academic papers in computer science and particularly in
machine learning were contributions are expected to have some sort of
practical application that is at least possible to foresee.
Indeed, a common criticism against papers in machine learning is "what is a
real-world application of this approach?". If you can't answer that
convincingly then it's unlikely your paper gets published, particularly if it
has rather mediocre results otherwise, as is the case with this paper (see my
earlier comment about the comparison to the two other systems which is a bit
of an anticlimax).
In fact, the paper (well, the preprint version) makes a clear attempt to
motivate the work with a practical application, saying that it aims to "learn
a simulator [for robotic agents] by simply watching an agent interact with an
environment".
> That is not true for academic papers in computer science and particularly in machine learning were contributions are expected to have some sort of practical application that is at least possible to foresee.
This is 100% not true for computer science and I don't know why you think it's true. There's are entire branches of computer science (e.g. complexity theory) that are highly theoretical and are certainly not geared towards immediate practical application.
Even in more applied branches of computer science like ML there are many papers that are geared more towards the fundamental research side of things.
> saying that it aims to "learn a simulator [for robotic agents] by simply watching an agent interact with an environment".
Saying that a paper could lead to a useful application down the line is not the same that saying that the paper is intended to describe the schematics for a specific useful application.
Well of course, it's not 100% true- who said anything about 100%? There are theoretical branches of computer science; but most of computer science is not purely theoretical. Certainly machine learning (and more broadly speaking, much of AI) is like I say.
I'm not sure who brought up a schematic? The bit I quoted is taken from the paper and it's a clear attempt to motivate the paper's approach as having a practical application. To clarify, do you disagree with that?
A lot of the ML techniques we're putting to use today came out of papers from the 80s. These techniques weren't practical to do much beyond toy examples, but they were proven to work. Now, we're able to use those very techniques to teach a machine to identify cat videos with 75% accuracy using unsupervised learning! [0]
It just means some people poked a pile of a bazillion if-else statements long enough, until the desired result came out. I believe everyone here knows the xkcd I am referring to.
Still, I think it's an interesting result. But, as someone on Fridman's podcast said the other day: recent advances in AI have all been thanks to advances in computing power, not any new ideas. The future if AI seems to depend on Moore's Law, more than anything.
Because it's a pile of if-else statements shuffled until it was able to perform a single specialized thing. Any housefly would by a thousand times as "conscious".
I assume that a housefly does have a basic form of consciousness. It's not going to write Shakespeare but it probably wants to mate. (I know people who's life goals don't appear to extend much further).
A) It's no longer practical to incorporate into real products given the hardware costs for training and inference. In NLP, large transformer based models can easily occupy a 5 thousand dollar GPU at a paltry 50-60 words per second processing pace.
B) The Research often demonstrates things that existing non-ML toolchains were already reasonable at doing with a better result, and a faster iteration time.
This demo looks like it's replicated the core mechanics of pac-man with worse graphics, 3-4 orders of magnitude higher resource consumption, and undoubtably many odd bugs hidden away from the pretty demo. It even needed the game to exist in the first place! While this is fascinating from the technical and academic perspective, it's hard to be convinced that any gains we see in the field won't be offset by increased hardware/latency costs from a product standpoint.