Panel Discussion: Investing in AI: Balancing Hype and Real-World Impact
Gen AI Summit 2023 - Open or Closed
Emergence Capital: Jake Saper, General Partner
A16z: Jennifer Li, Partner
Fellows Fund: Alex Ren, Founding Partner
8VC: Bhaskar Ghosh, Partner & CTO
Ficc.ai: Charles Elkan, CEO; Ex-Global Head of ML, Goldman Sachs; Professor of CS, UCSD (moderator)
SUMMARY KEYWORDS
data, ai, model, application, workflow, job, companies, startups, api, customers, important, product, question, infrastructure, investing, software, work, gather, areas, content
SPEAKERS
Jake Saper, Jennifer Li, Charles Elkan, Alex Ren, Bhaskar Ghosh
00:00
and CEO effect.ai An ex Global Head of machine learning at Goldman Sachs on investing in AI and data
Charles Elkan 00:37
Alex Grande and I was realizing this and organizing also, I think we have an exciting I don't know what to make out of last week called Office from four very different on important to venture capital funds here in the valley. introduced very briefly let them introduce themselves so they follow if they want to and we have some questions to dive into but hopefully the conversation will go in some directions that are interesting and useful to the audience. I mean, the start of this for me, so, on my furthest left we have Jeff CEPR. Jake has general partner in a moment capital very experienced investor. Then we have basket Gosh, a beachy. Beachy is general partner HPC. And then, Jennifer Lee is in the middle. Jennifer is a partner at Andreessen Horowitz needs no introduction. And finally, to my immediate left is Alex, who has a really fascinating career in the technology industry and most recently as the creator, the founder, founding partner of federal spies, which is bringing a new model to venture investing in technology. We'll jump right in with our first question which is for each of the Great Investors here. What is the what is the primary focus of your investment strategy? Maybe we'll just start in the way we had to learn so starting with Jake and then moving towards be
Jake Saper 02:39
sure. Thank you, Charles. And thank you, Alex for putting us on into zoom in HubSpot for CO hosting. It's a special conference for me. My wife got her start first career her first job with HubSpot. She was a sales rep. She herself is NBC years later. And zoom was the very first investment that I led diligence on when I joined divergence nine years ago. And obviously, we feel very lucky to be involved there. So it feels very cool to have this company to come together and to be on stage here. In terms of what our focus is in emergence all we do is be the software investing. That's all we've ever done. It's all we ever will do. We invest in companies like zoom. If you think about AI, we're trying to think about ways in which AI will augment workers going forward. So how can we integrate AI into applications things like what students doing with their sales like key product to help coach workers in real time using data on how to do their jobs more effectively.
03:44
Faster, he thinks.
Bhaskar Ghosh 03:47
That's Alex reporting it and thanks to zoom in not for profit organization for showing up. This is a kind of a sentimental rule for me. I remember doing a lot of organizing talks during my LinkedIn days and nowadays, both adults. Thanks for bringing us all back together. I'm partnered ABC which primarily the same software to the finish will be focused on AI data and AI you know, AI This is not one thesis, but the AI are based on everything we do. We have done a lot of articles as investments in other Baidu logistics with AI appears as you know, automation can zones for use of data and data most I myself I focus much more on the on the infrastructure, the platforms to load data into the AI and cloud and we are interested in AI infrastructure and the middleware and the data storing all the plumbing of AI. And increasingly with the advent of generative AI hitting mainstream also looking at which departmental application is for the vertical or horizontal will be highly accelerated by that with the foundation world center. Those are some of the
Jennifer Li 05:10
right on my part for you. So prior to joining me as a harlots as a muster, I was actually I'm a product side building developer tools and I like doing using some of the prior models to detect anomaly detection and that actually wrote was one of the earlier panelists Mahesh, that's all we need to do with conversational AI chatbots in 2020 71, ATM so testifies on our site that I had no idea I was gonna change the world. Five, six years later, but now we're here. So since I joined Andreessen has been very much focused on early stage into the enterprise infrastructure and we'll see investments, which are basically the foundations that involve various technologies to deliver on experiences in a given layer. That equals sort of one, in other words level. So anywhere from, you know, record databases, to financial models to even applications that can help 10x productivity is sort of interest areas for me. And as a firm, we sort of go very broad as where it applies to file type to gaming to consumer experience and where you're excited about the future. Yeah,
Alex Ren 06:26
thank you, Charles. And I there's I spent so we finally found I'm up on the on the phone. And so our phone actually is abused by 25 boroughs. The four leaders in the space and the indifferent organization, Google, Facebook, and also all the telecom tech companies, including Charles is a one hour distributor fellows. We also have lots of firsts for audiences, they start up Jeff CISO at the time and stuff. Where's the defense? We're here Roblox where's the DC Mall? And tomorrow and also leader in and even and, you know, obviously, one sec, for more to the Office of sit down. So we have a ton of our fellows that are deeply in the space and we must seem to both AI applications and infrastructure. And the four occasions we look for best AI Tony, from each quarter though. Accounting, insurance, healthcare, pharmaceuticals and price all of the sectors are looking for the best AI. So we are investing in lots of lots of time recently. Yeah, so
07:45
thank you, Alex, for everybody. I think now that we dealt with people's backgrounds and interests or investing, we can move on to the first question, which we prepared for this panel. So many companies are now building API's to make generative AI available and then many companies are building applications on top of these API's. So that the natural question for investors but also the founders, of course, is what if you're building an application on top of the API's, what are your notes for that application and then see most of the API's and other companies really good. He is also one of the votes for your API.
08:29
I'm very sorry, but this one so the application layer and software testing now everyone has access to the great technology products providing open AI from here all these additional models. So you're building with this this technology, the question becomes obviously what's defensible, what's terrible. The way we thought about this emergence is thinking about it using clay Christensen's job to be done framework, which is to say, what is the job that you are trying to do with this technology? And it's certainly starting to ask yourself the question, what portion of that job to be done could be done by going directly to the LM? And if the answer is the majority of that job to be done, could be done like I'll be straight such as GPT or to any of these other services, that it's unlikely you build something that's durable and defensible. The reality is many, if not most jobs to be done within the enterprise requires so much more stuff than just the model. I think we've kind of gotten we've lost sight of that with the hysteria that's taking place around generative AI. But if you're building, doing legal contracting, and you've got you know, a model that can help you edit your contracts using AI, that's really valuable. But to actually do the job to do all the contracts. You've got to build something allows you to edit text, well, you've got to build something that allows you to get the permissions to do certain things. You've got to get some sharing and collaboration functionality. You got to get approvals. Also, we have to have some sort of signature tool. There's a bunch of like, boring SAS skeptics that we find really exciting and find important that is required to build something that's more defensible. So we've lost sight a bit around the core, the MLMs and of the broader kind of SAS suites necessary. So step one in terms of defensibility is basic pro workflow to help get a job done. And there's no easy shortcuts to doing that. Well, so it's building great software we always has an incentive to is once you have that workflow, and you've got people actually operating that software over time. It's gathering the business outcomes data from that workflow. People have talked a lot about the proprietary data in the context of general AI. Our belief is that the most valuable proprietary data in general AI will be business outcomes data. So if you're doing legal contracts, it's not all the contracting data that goes into the necessary price. That is what happens with that workflow. So how quickly did that contract close? We'll use a certain type of clause that is generally on enabled using that business outcomes data to create a closed loop, improve your model and make better recommendations going forward. We think ultimately will be kind of long term disability.
11:07
Agreement too room is just remarking from an infrastructure point of view of infrastructure that applications are built on were never or as much of a differentiation or application installed, like which database you're using under the hood, as well as provides experience and guarantees that the users demand and have integrations and the Hudson models that consumers are expecting to, you know, complete the job to be done. Oracle is actually what matters to really prove. Whether it's business users workflow, whether it's a automation or even for consumer like that's where application value really impressed. So I know the title of the question is like, with all the hackathon sees things, just get on top of open API, there's a product ready the next day to use a series of areas a lot more, which I think is the most exciting part for what these API models are bringing is the repetition of improvements on the user experience on application layer can be so much faster, and we can create so many new experiences that the b2b feel or even because the apps cannot imagine before, such as you know, a lot of the conversational like AI or was calling or English generation models that can actually apply these inspirations or tools into the day to day self knowledge workers. So that's something
12:34
I think, Jennifer is a job application and we'll go to your on the inaccurate what is failure here is large enterprises, which has pretty good offices of customer data. Probably text and unstructured will benefit you can go back to section we're gonna start up who they're not yet here. To give an example like that. The last seven years, I just haven't talked to people from ServiceNow Zendesk, Salesforce, they should go and look at what's happening in source of good stuff. Then they have a lot of workflows. So the workflow part and local local part of this company is extremely important. So what needs to happen there? Well, using a language you can actually probably generate these workflows more easily to them, property Cubetto. So individual cases, those large companies are now going to start a call company for that. That's a good question to ask, because that could be a more I think we I think we see are interested in that cannot either is a very important area is marketing copy and debate on one important area. I don't know I can show that a bunch of hard companies that don't know my colleagues were sitting here. We are more interested in what we call artificial intelligence. Is that what customers expect? Is data industrial formatting an important problem? Is strong engineering of specific scope problem or a general problem? Can you plug in for the other stuff that you complete in which we do not get into our regulations and privacy in this problem? Either there's a bunch of recipe videos which are going to come up in the comment area, at least if you see the hybrid persona you haven't invested yet, but you're looking but in the act, you're coming back to it. Because investors have to be very judicious about what sector you're going to if you want to build a more than the actual workflows where you probably have a lot of first party data.
14:47
Yeah, I agree with three and and it depends if you're gonna build your colony based on infrastructure for infrastructure or locations. So like the most efficient mobile sites really confusing, because if you look from a technical point of view, UI use API is something that you don't have to select our follow normally content and wondering where's this actually company? Whether they know congress, or the best way to do this, though, isn't able to, you know, achieve the promo get paid as soon as possible. So integral programming cafe, so that means that you are using OPI or astropay API and quickly and you don't have domain knowledge, you're understanding the product and the token. So then the new that your product quickly, and I would factor that attraction and the paid customers, right. So then you don't need to go to your mother at day one. So you can use API right? So definitely for sure. And if your infrastructure companies that they love, you need to have your data your laundry, so always different a different situation. So that's what I mean. It's like I get it because the tragic material things and you probably need to hire less people use API and build that AI into your product and deliver to your customers. That's our that's my number. One space in the context of what building AI application companies.
16:17
Thank you. I think our last question actually already and I've touched upon it already in one way or the other. So maybe your artist will be short or maybe you have a lot more to add. The question is, what is an effective data strategy? We've all mentioned the importance of data. Large organizations often get data but most startups likely don't already have proprietary data. So what is an effective data strategy, but AI startups
16:51
where you can start with workflow, or you can create proprietary data by adding a workflow product, you don't necessarily start. There's a cold start problem for anyone who has that data. You can start that with like a living workflow product and gathering data and tying that to to outcomes. They're their creative. The deals I'm seeing going on right now for startups. We're starting to approach legacy companies and try to do a deal with them to get access to their data. There could be some sort of investment that takes place there'll be some sort of fresh air there's going to be dangerous but there are lots of ways to get access to, to proprietary data. But I do think that like there's an over focus on that in the startup stage. Right To me that was that was a mistake. I think there was maybe like if there's one you're gonna ask me like, what the news is good question for the panel. And let's go up to the one thing I changed my mind on with respect to AI today versus if we were doing this conversation four years ago. It's the criticality of proprietary data upfront. What I mean by that is I did not believe that it was possible to build a generalist model that was so accurate around specific to solving solving specific problems like you told me four years ago that a general sell, I've given them an LMS that point but before if you're talking to them at that point could you know solve solve their past the bar, you know, get a medical license, etc, I will lead you because I thought we need to build more to this specific stuff. And the reality is like a lot of these journals models are so good now that you don't necessarily need a ton of lighter data to get started. So you can continue to start with the general model. You can build a workflow tool that was more specific data, and then you can build your own model right here at any time you can find to an existing model setup, but you have options that you take years ago.
18:46
It's a very interesting comparison about four years ago, because I remember when I was at the startup, the potential is always we have proprietary customer service data. As I can train a better model than others, but actually in the general sense. Again, the floor is a lot lower. You don't have to start with having to gather a bunch of tickets, a lot of call nodes, the generative model capabilities. Pretty good. It's more alerting targeted towards how to fine tune or fine tune the model itself to serve this special purpose. I'll give you a couple of examples that I was very impressed by like using a generative model to create a very peaceful experience, but there are a lot of designer tools trying to go after Adobe has done a really great thing, but it's a design experience when congeneric A lot of you know graphics or photos, but there's just one company really focused on just texture generation because that is the component needed for a lot of game design. Either or a lot of, you know, product, Photoshop. That becomes a component that you can actually fine tune and build a lot of value as again, a component to a larger ecosystem. And that's where I started can actually shine by, you know, cleaning and gathering a lot of this unique, like purposeful data to really fine tune your own model to be very good at generating all kinds of textures, as well as enlarging texture to the element size to fill the screen, fill a billboard fill like you know, a flatbed or whatever, but that's actually where I see a lot of the current data strategy can be applied to startups. as not having to gather the enormous amount of perfect data but actually be very purposeful thinking about what's the data that's going to contribute to this vital order.
20:40
A lot of those problems subject so what followed mechanism so much I would say startups trying to build amazing with the new models is probably a hybrid business model. The rest of it, I think Chancellor can take on I think the thing that I'm surprised by all machine learning is how little extra how good the models are already, and how little incremental value and additional data so I think, you know my mind goes back to promise a lot like if you want to go work with first party data customers, what can you solve that they can't really use the customer's data. But how do you add a Delta of value that is so high on the upside for their product? And I haven't made up my mind as to what will happen I think so. Don't want to make any pronouncements. But I keep going back to how you would a little work in that context. How will they that helps in that context. So a lot of it I think comes back to not just how much data you get extra but what you have is actually pivoted and is enough plumber grounded things cooking. So pretty boring answer matters.
22:03
Thank you for the answers. And I think that's the theme of this panel is investing in AI so let me continue with another question about the environment for investment until we heard Alex this morning mentioned that he recently returned from Germany where there was a lot of discussion around regulation and the arc and in the financial industry where regulation and governance compliance is almost the first thing people think of when they think about almost any new technology. So my question for the panel and your LLM are so capable already as we just heard and a lot of research around safety for for Anaheim's but look, I did some regulation being opportunities for startups are the obstacles for startups, especially around privacy but not only around privacy. So how do you think about compliance regulation and governance?
23:06
I view that as good. And so most of us saw Sam Altman on Capitol Hill last week asked me Do you ever wait, it was an important moment in history of tech as we sit here in history Tech Museum. But I think it's important not just for
23:20
the ninth as you pointed out the city with the Actaea and the headquarters below us. Yes, literally, that if you make to this museum before, that was an amazing point in my technical packaging.
23:32
But the reality is the regulatory element when it combines all of this, it's important not just for the sake of humanity, obviously that's the most important thing, but also just in a much more narrow sense in terms of getting the thing or building sold successfully. I think that we are in danger without more compliance around the stuff of having an FTX moment in general. And what I mean by that is there is likely to be some sort of big blowout where some enterprise was using some model and didn't put proper guardrails around it and something really bad happened. So data was leaked, some financial transaction took place, it should, etc. And it won't be because the fundamental fundamental technology is bad is that oversight wasn't there, which is, you know, arguments anything that happens, the FTX situation. So without better guardrails, this industry is not going to have success in terms of taking off. Enterprises just will stop mining, right a bad thing will happen to some famous thing. We also started a few weeks ago three Samsung employees coming from a very confidential meeting, took those notes, dump them into chatting to him summarize, and then all of a sudden like yourself out of those those notes. So that scared one that rises from purchasing more stuff and they're starting to ask more questions, there will be a much bigger blow up and that will cause even more concern on enterprises and so my belief is that this compliance stuff ultimately is not going to be driven necessarily by government. I think it's going to be more likely to be driven by businesses saying I need to I need you to be compliant before I buy your thing,
25:07
you know, that the government would have before and I would say that I have to. I want to dive into this panel about when you have a population of jobs created or perhaps a society. Those are important things that you can dive into but not for me right now. I would say what we've already seen in the context of compliance and governance in b2b is already a very important no privacy is a subset of that. But you can think of it as industrial privacy. If you are calling you're building a product inside and the quality is the quality of the API's. What data is possible to outside with two different UPC very important number one, number two, you want to build a model yourself and ingested third party open data. How do you make sure that you're provably not taken cooperated with that person? And you should find out that you have done it. How do you go back to your mom? These are very hard problems, not just process by spectacle. How does if you want to get rid of service, our business did look at Bystolic marketplace is taking off? Because the data service or the business model is trying to build on top of the warehouse. Assume that the data that we're sending to customers has a blend of first, second and third. How are you making sure that you will absolutely know that people reading on Hacker News, a lot of stuff will come up there. And the b2b side of combative integration is already mentioned important and very much a blocker for large companies get smaller companies who are building those workflows or have been helping you want to come back to a company can decode a new subject. We have a suspicion that we feel at ABC that there is some opportunity there. If you decide to complete it.
27:18
This is my favorite topics and that was
27:24
so I think the issue is really long term. I think revision is good for the short term. So eschewed like, you know who was actually dominant or the shadow standards is deeply based on that start with a question. And then they do have some agency to initialize it to startups so that our poverty slowdown allows for innovations right before we haven't really like NCIS though, right? So I think this will be far away for this discussion of the design information stuff.
28:03
And I have a lot to add. I think everybody's covered. But moving on
28:08
with the theme of investment opportunities and challenges. I think something that those of us in the tech industry sometimes don't think about enough is how so much of the economy is physical in the real world. And so the question that I want to ask is, why can general AI provide value in the physical areas of construction, manufacturing, logistics, retail, transport, all these areas of the economy that are not immediately attack on water? The opportunities for journalists today are?
28:54
Yeah, and so we have the tracking also, you know, SLUBs robotics and we are AR and a lot of the people all the stocks are great to reward. And what I learned is that the biggest innovation days, neither product like remodeler, which is energy read to align manual reprocessing X percent Ceanothus. Right, especially in the content generation, so on then, so, from that angle, like, if you look at the last 10 years, like yes, looks like anything related to self driving cars, robotics is hard to make profit, right. So we look at those and Marjorie is really small. And I think it's a little bit far away from this kind of real application of our modern language models into this real world. Consider escapes Castillo data is interval training in robotics. It's hard to have those valuable data, right? So and the evil today is sort of driving our industry. They're not making money is losing lots of money, compared to like applying larger data models into enterprise applications. Content generations. You see, solid in that space are making lots of profit. So that's, that's my
30:23
I think this is a conversation a lot of coffee also covered it's very interesting. The Florida models where the gender identity I actually not replaced by disruptive tomorrow, white collar workers job then actually the regional manufacturing like a lot of the blue collar type of jobs. That's like one of the dawning of this impact. There's still a lot of applications can be made of course tours, you know, document processing as needed for, you know, logistics transactions, a lot of the loan processing or even retail says how do you actually present our product? How do you actually, you know, provide more authentic, personalized experiences to the customers there are many, many variations of applications that have been part of this, utilizing the facilities today already. And of course, there are a lot of research being done on how to apply this kind of intelligence towards robotics and how to generalize that to more human level capable robot to perform even more, you know, generalizable work. So I think there's just a lot of opportunities to leverage it, but today definitely is not yet the case we're seeing all of the big data are seeing sort of applications of abject failure.
31:51
Your answer which I gave which is like next gen RPA. will probably Polish most of these. It is in the traditional areas in the back office, shown the Process Automation where lots of unstructured data are limited. We expect supply chain there will be significant amount of those already I think most will get the multi modal model within part for audio video image. I'm not an expert in it. I haven't seen it yet yet. But it's ready for primetime for very complicated content, but I might be wrong. And we haven't focused on that IPC, but the boring stuff around Rpa 3.0 for all professional industries area. Now with serious software companies, they're yet to be seen. If you look at how do I buy automation anywhere in which other traditional companies should have done, the more significant I don't know if customers that that happened, right. So if you go in, but it wasn't the place of business processes, somebody has to write some pretty valuable software that we will use everywhere. This application. software written has to be good, but we feel there's a pretty huge opportunity for traditional industry. That's the boring part about it.
33:21
I can give Just two very quick specific examples of construction. Ai, like in the field. One company where we work with it's called drone deploy, and drone deploy sell software that helps farmers construction workers and others the people that work in the field fly drones autonomously gather imagery data and make decisions. And some of the decisions that may include things like helping people estimate how much progress they have, so how many stocks support or field they help construction sites identify with this variance between what was supposed to go wrong. And that has gotten so much better in the past six months is integrated in some of these few models. So it's been really cool to see that take place where we're another company called Rishi which does which uses computer vision, manufacturing, or standard workers understand how to assemble the thing and trying to assemble more effectively. It's also been super cool to see the impact of some of these models on the warfare. So I do think there will be real world impact as I read the title of our department.
34:22
Yeah, thank you, Jake. I
34:23
think it's been great to hear these concrete examples of new technology and improving productivity in the real world which will benefit all of us in society have one phrase that other panelists just used just as content generation, the other ones and then of course, the distinction between blue collar and white collar work? Maybe my next question is going to be a little bit provocative. You're welcome could argue that at least a significant amount of white collar work is more zeros, even an arms race and not actually improving productivity economy. I'll be very concrete Supposing your Home Depot or Lowe's and you're sending a lot of advertising email to your customers. And content generation can include the quantity and the quality of that advertising. But is it an arms race? Is it zero sum is there? So where were content generation and quantity and quality of content is going to really increase productivity as opposed to be zero so?
35:38
As far as writing so with that being said, like, I feel like there's the summer as far as the other big.
36:06
Day
