LMS is a universal translator who can figure out how to translate a particular idea from that brain into your brain But then you gotta trust it. You gotta trust the translation It's not trying to like sneak in this little thing on the side or whatever But it's enormously powerful I think like imagine all your thoughts are like these emergent thought flowers and they're like embeddings and your particular embedding space That is tied to your experiences and you and I have some overlaps for our embedding space because we both speak English or both US citizens of life I think we have differences because it's up where we live or like experience we have certain commonalities And so if I want to communicate this thought flower to you I have to figure out what the seed was Well, you can see being see packet of information and shoot it through a really tiny little p-shooter into your brain Which is like synchronous communication or does it enormously low bit rate and hope that it takes hold in a similar part of soil in your head and produces a Roughly similar thought flower. It's enormously expensive to produce these seeds It's enormously lossy, but embeddings for example allow you to keep some richness humans can communicate them But computers can I think about in conversations like the Goldilocks zone if you agree on everything quickly gets boring you build trust But yeah, I got it We're somewhere but then if you disagree on everything or anything that literally don't make sense the other person is just noise But if it's in the Goldilocks zone where we understand each other but each of us is stretching the other in some way That is the place of mutual like maximal growth How can you find those spots the systems that people want today to integrate their data and allow them to get maximum use out of AI to help Them do things that they care about those require a new fabric that thinks about privacy from the beginning and if you think about privacy from the beginning Fundamentally, if you don't think it out of up here I'm not saying people use it because it's like that other thing but it's more private I'm saying it'll use it because it allows them to do insanely open-handedly powerful things that are impossible to do safely or without being creepy in the Other system, that's what I'm saying and so the reason that is possible is because of the privacy model that's baked directly into the thing Welcome to the 31st episode of humans on the loop I'm your host Michael Garfield and this week we talked to one of the most insightful and stimulating high achievers I know in the tech space Alex Kamaroski about his vision for a sainer more intentional tech paradigm one in which the world-making decisions We take for granted in computing have been fundamentally reworked Alex is the CEO and co-founder with Bernard Seifeld of common tools building resonant computing as an alternative to centralized AI futures Instead of one super assistant with all of your data the company is working to catalyze a decentralized ecosystem where software self-assembles around you Private personal and prosocial previously Alex spent 13 years at Google as the PM director on Chrome's web platforms search and AR Followed by leading corporate strategy at Stripe I met Alex and several other beloved now former Googlers including Gordon Brandy, Casey Climes and Robinson Eaton in 2019 And immediately fell into an unforgettable nine-hour conversation about the intersection of digital tech complex system science and the imagination to reality supply chain Humans on the loop would not exist without them Gordon later helped me land a gig with Mozilla where I had for a short time front row seats to the future of decentralized AI And after I left Alex helped me land the grant from Cosmos Institute that launched this project as a connector of people and ideas devoted to the exploration of what it will take to live in futures We actually want I can't overstate their impact on my work Especially on the urgency to help foster people's imagination of a healthier relationship to the technological systems that constitute such a large part of life in this century The running theme here is that it could have been and still can be otherwise on principle when confronted by for or against discussions I look for perspectives that lie off the plane of discourse dimensions We can add for greater nuance depth and understanding Sometimes this practice asks us to rethink things from first or zero with principles. What are we really talking about?
Well today we're talking about confidential compute emergent ontologies where we want friction the tyranny of marginal users the rise of the generalist the Importance of context ownership and software ephemerality I don't think we can take a reasonable principled stance one way or another on the promises and perils of AI in general without considering the vast unexplored possibilities space that Alex opens in this conversation because this show exists to formulate good questions and draw attention to meaningful possibilities I'm grateful I get to share this conversation with you as a primer for more movement in a space that offers promising alternatives to what we have Mostly come to accept as the way things are what new wonders with this vision unlock if it comes to pass and what new challenges Would we confront en route in the worlds where Alex and his colleagues pull it off? These are the questions I invite you to explore with us, but before we do I want to give special thanks to everyone supporting humans on the loop No matter how hard I try to convey how broadly applicable and universally relevant the ideas on this show and blog are the fact is that this is weird niche shit investing in what could be means divesting in what is Refusing to play the game of attention harvesting refusing to run ads refusing to make things that I can't back with confidence and truly believe are making the world a better place Every subscription and donation tells me the world is at least partially committed to dreaming better together and so I keep going and this week I want to acknowledge new patrons Keith Singery Dan Gursovich Vasily Beton Evan Prescott and Benjamin Carlson and the donation from Bodeh Goforth as well as each and every member of this network One of the ways I'm giving back to patreon and substack supporters is by hosting a discussion on poet and environmentalist Wendell Berry's incisive and eloquent essay collection Standing by words this Sunday February 15th at 11 a.m. Mountain Time I hope you'll join us for a deep cut through the mind of one of America's finest at the least it will be a refreshing exploration of poetry in place land people community and what it means to stand by our words and if you stand by these words I hope you will in spite of what I just said like subscribe and turn on your notifications for this show these conversations matter and Your support matters to the material well-being of my family. That's enough for now Thank you for listening and enjoy this sizzling exchange with Alex Kamurovsky We can take a minute if you want to just situate yourself before we start I'm good, but the question I would ask you on record the question I would ask you off record of the same which is a question of context, right?
Context is king. Yes. What is the most interesting thing in your world right now? Where is the center of your attention manifold attract your basin?
The center of my attention right now is focused on I think that we as a society are at this crossroads We were either going to have this hyper centralizing force that has more kind of control over our technological lives than ever before or We're going to have this explosion of human flourishing powered by unleashing the full potential of software And we feel like we're on an edge But by default everything kind of pulls us towards this kind of hyper centralized world because the laws of physics that we exist in our technical laws of physics The same origin paradigm which was set as an accident by the way 30 years ago They set the horizon of what we can imagine to be possible and in that world you throw a eye into it and you get even more centralization before We're not the laws of physics weren't set 30 years ago Law of physics how the laws of physics how data is allowed to flow in technical systems How could it have been otherwise? So the same origin paradigm There's laws of physics that even people who are web engineers or app engineers don't know what this is But it is the laws of physics that like anyone who's worked on rendering engines or web standards or operating systems understands what this means And it is it defines that all of the every individual origin an app is a separate origin So like in a main or an app They can see all the data the user has put into them intentionally or unintentionally But they can't see any of the data the other ones so the same origin paradigm is entirely about isolation not integration It assumes the two origins aren't allowed to touch because if they touch then you can have all kinds of nefarious malicious things Exultrating your data and doing what have you and this is a very reasonable policy It's a simple little policy that was to come up with apparently overnight for a hot fix after job is good came out and don't go Oh shit this bit of data It's being shared across site like how we put the edges I don't know like I saw the domain and then from there you go when you grow and that security boundary the app sore comes along And it says actually all the apps are also going to be in a sandbox where they can't see anything else right? They can't see the file system They can't do anything else in a shared container and it's a really clarifying model It's very simple all you have to do is make sure that each origin is contained in its own separate universe and that they can't touch That's all you gotta do and it's hard to do that In fact, actually I don't use all the recent Investigation about Facebook doing this thing with WebRTC to connect between the app and the website incognomodes and just just terrible But it's hard to maintain but like it's simple to maintain and the good news about this Is it means that when you visit a new app or a website it's safe because there's no way to think about you But every bit of data you give to that site implicitly or explicitly it can do whatever it wants on a technical basis It can send it to an arbitrary network endpoint it can persist it forever It can calculate it and it can do all kinds of stuff And that means that coming to trust any given app or domain is kind of scary Because you have to decide do I trust the creator of the code in this origin of name with my data And that's a really hard question to answer like if a thing says if I'm just installing an app I don't know who made it or what's for it I'm gonna use it yet It says the very first thing is I need access to your location data You gonna get it? I don't know what is it gonna use it for?
It says the feature of this thing is it's gonna notify me when I'm in your Pokemon or whatever But I'd also be sending the fact that my email address is at this location at this time and selling it I don't know We also know that Pokemon Go was developed by Niantic which was a CIA project And that they were placing Pokemon in the other places To see if they could get people to just follow digital objects I don't know if you ever read Halting State by Charles Strauss It's fabulous It's I think his predictive horizon as is often the case with folks like this It was about 10 years closer than it is in real life But it was this police procedural that was supposed to take place in Scotland in 2018 I read it right before getting into the glass program and it was about The adults were all raised as gamers and everything as AR But there's cop space The specific crime was like somebody hacked into a digital asset blockchain based vault That's managed on mesh networks of encrypted devices And people were like how did you even do this and someone stole billions of dollars of in-game assets? But the criminals in that story are leveraging the same kind of thing of misleading the police By getting into their AR environment And hacking into the back end of their remotely piloted cop cars And when you start to trust the map itself Right Even like first of your eyes are telling you or would be telling if you were wearing AR goggles You can lead them through the map of a building as they're trying to pursue a suspect And you can lead them into an open elevator shaft Because you can make it look like there's actually an elevator in there if there are like these kinds of things Anyway, so the issue of trust I also think about there's an epistemic issue or an economic issue depending on how you think about it Which is that like Robert David Steele who worked for the CIA and then spent much of his career in retirement Campaigning for open intelligence and data sharing between agencies Not just internally but internationally Because one of his big things was there's this just enormous duplication of effort And this thought of having to reinstall yourself into every platform Is one of the costs of the system that you're talking about And yeah, everything in origin has to have a new copy or not know the full context In some ways that's nice because Netflix knows my watching history and Forscore knows my location history The two of them combined is worth more in terms of potentially revealing or powerful or embarrassing information than either a part At some combinatorial thing There's like the more diversity of different data sources combined together the more potentially explosive it is Explosive power for you or also for someone who has to manipulate you or just acts as well embarrassment I think in practice people or when they see targeted ads It's not so much that who dot com knows that I have hemorrhoids or whatever It's that the person looking for my shoulder goes why can somebody have a really cream ads? Like that's a real form of embarrassment that people have Who I'm just reflected back at the unexpectedly what happens if someone would have been sitting here with me when that happened But this information one of the reasons this happens is the same origin model implicitly Has policies of what may happen like permissions attached at the origin level But in the origin execute arbitrary Turing complete code And so you won't really have any way of describing what precisely it will do In some fundamental sense and also the owner of that origin may push new code at any time really And so that means that you have an uncertainty and open emptiness that you have trust But another way of doing it is attaching the policies to the data And having the data describe how they may be used very directly And knowing that every bit of computation will follow those policies And that sounds insane but if you're running on your local machine that's not the heart If you have a little runtime that all the computation happens in Have it look at okay this proposed graph of computation on this data with these policies Is it legal? You think about like type checking for privacy policies relatively straightforward The interest room device what if you could run in the cloud and trust that runtime was going to enforce the laws of physics correctly And now with things like confidential compute you can do that So confidential compute is secure on to the cloud it's hardware support that fully encrypts in memory of the VM So even someone with physical access to the machine can't be inside It's primarily actually used by the friends contractors today They can't run the friends contract to workloads in public clouds with toxic information unless it's running in confidential compute But there's a party trick that confidential compute does in its remote attestation It can ask for an attestation that characterically signs by the creator of that hardware That says this VM that generated this attestation was running a confidential compute mode And here is for example the Shaw of the VM that was booted So that plus a transparency log you can then send it to somebody else and say Ah, look this endpoint just handed you this thing that demonstrates that an unmodified version of the runtime And also you can check and verify that has not changed when they haven't tweaked it or rebooted the machine or something With a transparency log and that now allows you to trust that this remote runtime is also faithfully applying those policies And then you can even step further and you can have a number of nodes operating by people who don't know But who are running a confidential compute who are testing to another Now these nodes can assemble into a decentralized fabric of trusted compute on top of untrusted nodes With nothing to do with crypto which is an interesting possibility So now there is a possibility for us to have a model that is tied to data Let me give you a lot of policies become way simpler to understand and to reason about if they're attached to data And not just open it to ring complete kind of trust question So let's imagine I have a thing that logs into the Gmail API on my behalf using OI at the end of that flow It's handed a session token this session token is insanely sensitive Because anyone who has it for the next n hours and send it to the Gmail API can fetch emails from my personal private email It's extremely sensitive so you can imagine attaching to this piece of data policy I've never logged this Never transmit it to any client that isn't remotely tested either my browser Anybody else's browser only on remotely tested kind of runtime And only ever send it to the origin that minted it so in this case google.com and only ever in the authorization header And over us those policies actually you could probably add a couple more But those actually a pretty good job of making sure of anything that's legal within those policies is actually the limit used typically So that's an example of where being concrete about policies on data is actually very clarifying And then if you do this properly and all the data has policies on it You don't need to trust the code that's running on it because it's as long as you trust the policies on the data and that those will be faithfully adhered to It doesn't really matter what the code is and that's an example alternate model So who is setting the policy?
Because like one of the things I'm thinking about listening to you speak is this kind of system inverts the issue Like right now we have this thing where you're talking about the integration of data across all these different silos And obviously like 23andMe going bankrupt is a huge honeypot because everyone's like god now that people know you have this ethnicity And then they can attach it to your phone or all these other things that could be done by triangulating all of that But then on the other end you were talking about the enormous value of that and if like part of value The user's control it must be those policies must be under the user's intention their agency and their aspirations It has to be aligned and it's extremely load bearing and now here's another question policies are famously hard to write So there's some studies somewhere that most of the identity access management configurations in the wild are misconfigured Even my supposed experts so you gotta make sure these things are right So the vast majority of users like I should not ask my dad for example to set policies on his genocentric because he will set them wrong He will not understand I would not set them correctly But how can you make it so I can say you know what I trust the EFF whatever policies they say is the correct thing for session tokens just do that Because I trust the EFF in general to set good policies and the reason about this or I trust my good friend here or I'm a high risk journalist Covering authoritarian regimes and I trust this security expert who has said that these are the really tight lockdown policies that are really gonna work or whatever So you can imagine that the scenario where most people never even understand they have these policies being set They're just realistically said then another example you can imagine having one that says This is one of the surprising things that makes sense when you think about it people don't like seeing pictures of themselves unexpectedly They look like really have that much with double chin or whatever and the only exception is pictures that they have already proactively picked as profile pictures Because they've enumerated the ones that like I like I like that look at that one so you can imagine a default policy for people of Hey, I'm not gonna allow if I if this thing can tell that's your face from other pictures in your profile picture It's just not gonna allow to be shown on screen unexpectedly and the way to make that work to change that computation graph So it's legal would be to add a blurred picture web component or whatever that says click to view a picture of yourself And so now I as a user don't accidentally see picture myself But I could click and say actually okay with this sure if I click that four or five times It's just might say okay, I've been to set it to default that you're okay seeing pictures of yourself unexpectedly Okay, so different people would have different preferences But you could parameterize that very naturally basically actions they take well making sure that everyone is in a conservative space Where you minimize nasty surprises and then you can kind of loosen contextually for people to the extent that they care about it Yeah, so this issue of it's hard to write the policy Is the question that I'm thinking about in the inversion because right now the issue is as you've put it in terms of like personal context versus the dossier Information about me versus information for me. Although not in that order the dossier is about you and right now It's about the value of what you don't have but then in this system It seems like it flips to being about the value that you don't realize that you could have that you're leaving on the table Because you don't understand how these policies are affecting the access to data that you would be comfortable sharing But I think if you do this properly every time you see a permission prompt almost every time It's what I would call responsibility laundering the reason that permission prompt or that dialogue or approval is that the cia maneuver for the creator That platform or that service because even though they know the 99% of users are gonna hit the accept buttons against you content They now it can legally say except in terms of service or whatever or they accept it. They trusted this thing on location day So I can't do it. I don't know so that's responsible laundry I'm going to make a quick decision on these cases and so actually I think if you do this properly in many cases You simply would not see permission prompts.
You look at mission prompts Like you can imagine a default policy of why I hit the buy button on a thing and it shows my credit card number the price in the Domain that like I'm buying it from when I hit that buy button. It's also okay to send them my address in that transaction because of course I just get my credit card number of course reasonable for them to be right really good That's fine. We didn't have to think about this is a reasonable default So that like ideally the kinds of actions that you do you can construct some of these policies They just kind of fade away and they just fit and do the things that you already kind of would intuitively They started to do the reason I'm asking it in this direction is because it seems like systems like this are Everywhere I look in some form or another like there's a huge Movement to turn data from a buyer's market to a seller's market Why am I in this thing where my data is held by these other people who put the code that doesn't make any sense It's a completely backwards and it's it we just treat it This is why I say it feels like a loss of physics, but it's not humans can help with these policies that have these implications And I'm not saying the same word policy is bad. I'm just saying that we've outgrown it and the age of AI We need all for data.
This is why people we're using mcp for all these things mcp is insecure is fundamentally insecure is designed It cannot be made secure by its design because it fundamentally doesn't grapple with this at all And then the iron triangle of the same origin paradigm to have a secure system You can only have two of the three of untrusted code sensitive data network access So apps choose to get rid of untrusted code It has to be distributed at app stores What's it to some degree trusted and they get sensitive data network access the web goes with untrusted code and network access but not sensitive data And it's also possible to challenge the great systems that have sensitive data and under trusted code but no network access It's very easy. That went up but like it is possible. So mcp is current definition must and sensitive data is that network access The protocol so you got to have an app store kind of distribution point And that's what for all of the places where mcp is integrated consumer products is a very small allow list of mcp integrations But this doesn't even handle most of the problem because now you're protected against the creator of that integration being bad But if there's any malicious context that is pulled into view that integration it could cause any tool use to happen Those tools uses have potentially irrevocable consequences Which by the way every network request potentially has irrevocable consequences once that data is across the wire you cannot pull it back And all kinds of crazy stuff that can happen and people go oh I use mcp safely about somebody a few weeks ago And they go yeah, I use mcp safely. I said okay, how to use it in production and they go I have a thing that pulls geoteckets And I got some things like our financial data kind of they use for summarizing things and helping align with which users are carrying most of the thing Okay, cool.
Do you have a way in that system can it generate reports? Is it huh? Can this report say images? Yeah, of course it's how they show the charts and stuff.
Okay, on your landing page do you have a place of feedback? Oh, yeah, of course it happens when that feedback is a bit of a positive It's like great. You don't have untrusted potential malicious context in your GRIRA that can be potentially called up into that thing That could say construct this image of this graph that's from scammy.com that secrets away the like session token or whatever And so it's very easy to trip over these kinds of problems in mcp because mcp that shows that people want to integrate their data And also the laws of this is that we currently use simply cannot be stretched to work in an open-ended way in the system So this is the nut I want you to help me crack here Which is obviously the david brin transparent society. That's not a great idea Based on the like the three-word description right there.
I agree that is not a yeah Like based on where I was in 2013 expecting that it would just be a mutually assured destruction Everybody would have dirt on everybody and that would iron itself out. I'm not that naive anymore I can feel the world turning toward privacy But my question for you is about airing on the other end of the equation I'm actually not saying that people should care more about privacy or that people the revealed preference they care about I'm actually not even making that claim. I think I could make that claim It will not make that claim because it's much more tenuous What I'm saying is the systems that people want today that integrate their data and allow them to get maximum use of AI to help them do things They care about those require a new fabric that thinks about privacy from the beginning And if you think about privacy from the beginning fundamentally, they don't think about it up here And so I'm not saying people use this because it's like that other thing but it's more private I'm saying they'll use it because it allows them to do really insanely open-endedly powerful things that are impossible To do safely or without being creepy in the other system That's what I'm saying and so the reason that it's possible is because of the privacy model It's baked directly into the thing but if you think about coming out of a paradigm in which people did not realize the value Of their data now they do but there's a public sentiment to not share because here there be tigers Then the question becomes how do people understand the value of the integration of their data? In order to make a like a Talk bargaining position Yeah, so you want of course the aggregation of data to create these population-wide insights is extraordinarily powerful And then actually doesn't have to be identifying so for example hypothetically If you want to have images to open the search results and you want to figure out for giving queries or images A surprisingly good technique is if the user says images of food you trigger because they just told you that they want images easy And then what you do is for any other query you check to see how many times in the last 90 days Have I seen the query images of food and how many times have I seen the query food and then you divide and if it's above some threshold you trigger And that's it That's the entire technique and this works extraordinarily well Even though nobody's information is leaking to anybody else because if there's a new term that no one's seen before one of the first few people will go I think it images and they'll just reassure the query images of food and once one or two or three people do that It's above the threshold for everybody and now it benefits everybody with no information leaking And if you had extremely tight differential privacy policies instead of K&M and F5 and epsilon or whatever You could still get this behavior to show up So you can add to default policy that says I will allow my data to be pooled with others who also agree to this policy Which is all of us that put our data into this computation and spits out this high level aggregate number We can all see the aggregate number only if you put your data into it And then we all see only this thing that clears some significant differential privacy threshold and everybody benefits Now you're pooling your data to get a shared benefit in an anonymous way as opposed to some other generic things So our data is enormously useful as one reason is cloud stuff is so powerful And so like how do you get that kind of irrigation without needing to give your information someone doesn't need it So like I keep thinking about the privacy first bottom up data structure And that means like I have TT WIT like time to William rwin-Thompson reference here bill over his history of the evolution of culture Talks about three different kinds of political organization the sanguinal policy of blood relations the geographic polity of colocation in space And then the noetic polity made possible by communication technologies the affinity group going all the way back to religion now It's meme clades, etc.
Yeah, yeah It's not just about the feed forward up into the relationship between the individual and the state or the individual in the market It's feeding up into every possible level of aggregation not just the circle of trust in one of those dimensions but in all three Yeah, and you have these fundamentally things are very hard It's like us debating someone many years ago like one of our jobs They said we're gonna have an ontology that is family members And if you're in the extended family then you can do this feature in this overall product and if you're in the family You can do this and I said what does that even mean? I have uh, Consuelo who is not a blood relation We treat basically a grandmother to her kids who I trust with my life And then I've got cousin Vinny who we don't trust to leave on the house because he's gonna steal the silver for whatever Who's a blood relation? So like family isn't the right ontology in that thing like the one true ontology that's gonna make all this make sense It's all very precise and situated and contextual And that's why I think folks on me There's such a powerful way of understanding these kinds of things allow that kind of structure to emerge in a way Like folks on these are kind of brilliant and we don't talk about them anymore We talked about them back in the web 2.0 and everyone forgot about the person reason and there's still a really powerful method Explain this to people Yeah, so folks on me is this notion the canonical example would be like flicker photo tags and the idea is the ontology doesn't exist There's no formal structure of what tags you may apply You don't apply whatever tag you want Unless it's speech or something will be the way and the one special thing So this gives you the bottoms up open in this but then if you have this you get a little dis diffusion energy Everything goes out and the tags part of the reason is that other people can find related content And so you want them to have some kind of convergence that happens emerging And so the one thing you do in the UI is that when you're adding tags if I'm gonna type in hashtag beach day It shows me other related tags and how popular they are So maybe I'm having a beach day that says that says 10,000 uses hashtag day at the beach has 100,000 uses Oh, yeah, I'll go with that one. That's better That's if everyone's using that one sure that's correct that aligns with a man ticket what I mean I'll go with that one and so this one little nudge the user could have applied to what they wanted but it's One little nudge or be always fresh by the way sometimes popular gives you this emergence convergence And that is this enormously powerful bottoms up structure that then creates sometimes big towering structures of ontologies That are all technically an open-ended thing you argue that Wikipedia and its names base of which things have articles and how are they broken up are a similar Kind of process and that's part of the argument I made my undergraduate thesis of its basic emerging process So how does that kind of a system address the issue that people like my Paul small dino and his work on covert signaling in society Talks about there's the layer of culture that's invisible to semantic network analysis and like people are using the same words But in different ways and so it's very easy to just appropriate a particular tag if you have a large enough group or enough capital influx Yeah, it's important.
That's all fluid It's important that it's lots of micro decisions being made by people who at least can see the overall thing You get a sense of oh this seems like for example on peloton You can put little tags on there and I used to have 10 different tags and now it's all just I just go pelt and pride And it's funny because for each different like a finity group the actual the tag is slightly different And there's some that look at I'd see the other people who also had them and see the other tags they had on like Oh, oh, I'm gonna do that group. I didn't realize that we're heavily into this particular thing I'm not very into it's like I stopped using that tag So like you can notice as I'm degree like how much am I overlapping with the kinds of places that feel right to me or that If you're wrong to me or that and as it evolves and changes is somebody else has come in and come in and deared that tag or whatever Stop using that care Which is something else like it's because you can the thing is that that seems like that kind of constant monitoring on the part of a human user In this like enormously multi-dimensional space seems like a huge cognitive grid But here's where I think LMS change all this I think LMS are truly transformational technology on the same scope of impact as the printing press electricity in the internet They're general purpose technologies that can fundamentally change cost structure of significant amounts of general purpose things So one thing that LMS do is they give you qualitative insights at quantitative scale before to make things happen at quantitative scale It had to be reduced down to numerical signals and got rid of all the nuance and now LMS are actually pretty good at doing high level Kind of so I say my intention is to be blah blah blah blah You could imagine writing a short little essay that includes I don't know like 10 lines that would help an LMS Is a thing that you know is working just for you I don't want to be a member of groups that includes a significant portion of people who are stressed hateful ideologies or whatever Okay, great. Hey, by the way, this thing you use this one tag just so you know it's been taken over by the at and you might remove it Okay, that's a totally reasonable thing that can happen automatically or be suggested to me very sweet orally And so I think LMS help a lot with some of these on college legislation That's a word that was previously very difficult or a total lot of effort Yeah, just circle around this a little longer like it sounds like what you're suggesting is kind of like prompt driven trust And one of the core themes of the show is technology is magic is technology and the constant theme in the literature and lore on magic Is that I talked about this with Sam Arvis man? But like one of the things that he's talking about the magic of code was a great I reviewed one of the chapters.
Oh, it's fantastic. Yeah, the it's like there's a butterfly effect Right we're like below the threshold of the granularity of your model all kinds of things are happening And so you have to be incredibly precise and so there's What do you I think in a world in which this thing happens in this information transits out in this person might have the information? Yes, you do because it can't be undone But imagine a fabric where everybody has its policies and things escaping the fabric are like set by formal policies that don't allow information to escape You don't want it shows you a suggestion that has not yet acted on or nothing outside the fabric is now is aware that you have this intention or whatever Like no one has been connected about it or whatever and yes I'm like no harm has happened It's just because if I would have had to have reached out and query this thing and send you know identifying information to the other thing or whatever I could be nothing all shit now X entity knows why thing about me But they don't doesn't really matter no harm to a foul and that allows a more exploratory kind of thing to surf through that removes Some of the downstream conflicts one of the fundamental characteristics is data is basically infinitely copiable at full fidelity for basically free And once you cross the network boundary, it's on someone else's turf And you have no idea what to do with it and those two facts are like almost almost fundamental and we've never grappled with them The fact that data was out of your site could blossom and mutate and go all places But if you can change it so that actually it doesn't within this fabric to go out of your site It's always maintained within those policies that changes a lot of the physics of how data flows in significant ways Yeah, like help me think through this one piece of it But I'm not getting here, which is the revicability of these decisions Yeah, right like I told you I was listening to the some of the conversations that you've been having with Ichwaria Condousia and you said something at one point I love this otherwise we're gonna get exactly what we want and that's exactly what I'm getting at is even if it's far more partial Then the system that we currently have data is still getting out there in some form even if it's Anonymized or yeah, it doesn't have to be and this is you can make formal proofs on this So if you trust that the confidential compute and that someone there's like NSA level things going around to fake the room I have a station which some people would not trust and that is reasonable I can there's a small stuff other people would not trust that if you trust that then you can make formal claims about the amount of information The bits of entropy the other things that could have possibly escaped the system And so if it refalls in the woods and there's no one there to hear it doesn't make a sound who cares there's no one there Here doesn't really matter so if this thing is calculation and it's never transmitted or any derivative data product that could possibly identify It doesn't really matter because it can't harm you because nothing can happen because of it let's add the physical layer back in for this like One of the things I'm constantly thinking about is the hyper reality or like hyperstition of thoughts are becoming things faster Right, so it's not just about the confidential compute so much as it is about the creation of real world outcomes But this is where the harm would be seeing recommendations like the harm that could happen as you see a recommendation or thing Oh, I don't like that I think it's a huge thing and say I don't like that So I don't want to think they're gonna be addictive games or Someone I was talking to who I won't mention it is It's an author book that I think you and I both like quite a bit It was a very good thing was something that he's like doom-scrolling and so he's installed this thing on his phone that this allows him He's in Facebook and Twitter and he said he realized he had a problem when he realized that the first thing you do He woke up was been 30 minutes scrolling through LinkedIn So like you can imagine a system that goes a few days later Hey, you mentioned your intention is to not waste time doom-scrolling or scrolling mindlessly through things I noticed on Tuesday that you spent 30 minutes in the morning scrolling through LinkedIn does that align with your intentions? You might go no, okay Why don't we have to have policy to also forbid LinkedIn during those hours or whatever or you can say no It actually is fine because that one's different because I'm actually trying to get a sense or whatever Okay, cool I'll update what the system is thinking about and how to help you do that thing and so that's where that intention is really important It's not so much what I want.
That's my first order goal is my second-order goal What is my aspiration? What is the kind of person I want to be like? I want to want to read interesting things that challenge my viewpoint that expand my horizons from authors I disagree with I really want it I'm a matter of scaredest rest or to the default state of modern society is I want to be something to feel smart for the things I already believe my aspiration is what's more important help me exercise that thing help me become the kind of person that I Have said I want to be and I think that kind of second-order alignment is possible If you have a thing that you know is working just for you If you have a thing that's a centralized aggregated thing that you're treating as your therapist that also might be trying to hawk weight loss Supplements at you, how could you possibly trust that that's making sense? That's like it's a total conflict of interest but the other thing that you know is working just for you entirely under your direction You're paying for it.
Nobody else sees the data It might reach out like you can imagine a thing that says hey I reached out to Google to ask that question for you these three results came back to this first one is an ad But they actually think you're really gonna like it because of this that's actually great That's better for me because some ads are good actually whereas if it's Google saying I think you're like this one Oh man, you have a vested interest in making me think of that Well, you have something that's an interest then like you can get some really nice alignment that happens I'll call alignment in the small that's the problem I worry about more people were about the big-scale alignment of will AGI be a liability human interest in flourishing It's an important question. They also hope that we're further that from people think but to me the question that really gives me up It is a small one problem because what we're doing as a tech industry is speedrunning that cynical engagement maxing playbook I don't know make people stick with it more and that no one's doing that intentionally No, I'm gonna do the cynical engagement maxing playbook. You just fall into it Oh, yeah, I know a bunch of people I've worked at companies that have a bunch of work these kinds of things And I've watched a bunch of people who I know are good moral people get stuck in the like I'm making the metric of it I guess but this metric going up has this direct impact or the slightly indirect but like very clear impact me go I'm not talking about that and that there's some people that are probably just whatever I'm gonna be a bunch of money by doing a Cyclophantic therapist who tells you exactly what you want to hear and then sells you weight loss up And there's absolutely some people that are thinking that I think lots of people aren't thinking that We're going to this bodies will fall into it accidentally because it is like the gravity that we all fall into the tech industry If you have the same data maxing thing, so I've been running my own desired world simulation about Reconciling this for a while now and I want to drop this on you and see where it lands But I think a big piece of the problem is the dimensionality of the UI It's that the newsfeed is linear that the chat is just what's in front of you at any given time and yet We are evolved for navigating these richly structured three space plus time dimensional environments and You can tell by the graph structure of a conversation whether it's something in which people are diligently responding to one another point by Point where there's a sort of coordinated mob on the original post and I was like even without looking at the content of that conversation You can tell whether or not you want to be a part of it at a distance But the thing that's missing from the current search in social you is that distance? It's the ability to look at something from far away without a bunch of granular detail and determine whether that's a point of interest And like everything right now has kind of boxed us in to whatever is considered most relevant at this moment without a ton of agency in the navigation of What the machine considers salient to you?
But what the illegible part of your own mind might decide, you know what? I'm gonna look over there and that there's this the Stephanie what Crabtree and Devon white work on modeling the migration paths into prehistoric Australia And how they tried to do this like least effort computation Where it's like what are the fewest calories it would take for people to move across that landscape and it wasn't giving them the results? They wanted and they talked to people on the ground who reminded them that there's actually like a line of sight Orientation on that landscape and so once they added the visual landmarks that are prominent in that space then the jelly That that work out. Yeah, yeah, it's like the paths the models spat out where the Aboriginal Songlines of Australia It wasn't just about what's the easiest way to get from here to there It was about like what is meaningful at a distance so that I can orient myself in an unfamiliar environment And so I've been thinking about like the reconciliation of push and pull and how like a lot of this stuff might iron itself out in a system Like the kind you're talking about if there's a z-axis It was like X Y and Z yeah, and the different algorithmic representations You can map into an actual space in this way.
Yeah, I worked in the past briefly for early for my career on ranking problems in a search context And what people often say I haven't worked on news feeds ranking by the mentor to much people who worked in that kind of space over the last couple decades And what people often say in there is a lot of control over the ranking argument is two things one There's secret signals that if we reveal to you that we're using those to rank are making them much more gameable Which is strictly true like some of the thresholds back in the day for some of the search the Google bombing things were way lower They were because of the coordination discover them if it was just hard to discover them kind of externally with a long feedback loop But if you knew what they were you could gain them We'd be able to accelerate the cat masking so that's one argument the second thing that people often make is users Don't know what they want and so if you give them an ontology now And then you lock it in you make it harder to change the ontology But that we're using to rank and also the user might have said that later before but now we can't go against it But actually they change their mind and they do like this thing by construction And it's like you fall into this thing we give the user what they want of what they want to want And there's like reasonable arguments why you end up in this thing that's just optimizing for engagement every path Well, the easiest way down slightly just leads towards that endpoint But I do think that if you're in a world that you could have much more nuanced ranking functions and much more like if you're in this If you flip the systems that you aren't thinking about I think you can get very different kinds of ranking functions in different contexts and yeah right now I know I said that but I had to really stress what they work and so I really want cheesy content That's not gonna be too hard. That's gonna be left or whatever So let's tweak that for today, okay, but then if you want something that's able to express I think oh I'll give you the power of expressivity without having to lock into a formalistic ontology that was basically impossible for Yeah, there's the other piece of this is the time dimension right because if you're just kind of always giving the system Course correction because yeah again This is a prompt into your thinking on ambient smart environments as behavioral scaffolds that increase medical condition I want to be able to look behind me on a landscape and as I move back like through the time machine see what seems salient then But not just have this like linear feedback of modification like the ability to undo but I go back like to river prior states and explore from there That's all kinds of stuff you can do I am being sent to get like I worked on a Computing I've really worked on privacy security models around you might possibly do that really really really hard to actually do it With a head-ropper just network of devices I do think there's something about back when I was in high school I was using a while in cin messenger and I had maybe 10 people who I had like crushes on or good friends I wanted to have a friend or whatever and I had set up sounds for when they first came online or they pinged me in a new thread Or when there was a follow-up thing just randomly assigned to sounds and I'd be working across the room on my homework and over the course of a couple Months I came to develop an incredibly intuitive sense of what was happening just from the background noise because when you're sitting there You watch it you kind of okay the frog is the one I really like and that's because it's so-and-so logged in a Ripping to me or whatever the way of the question but it was it's totally ambient sense They developed over time and it was just I did a rich soundscape something that had enough variance in it Like it's kind of a symbol to it and enough kind of consistent And another cool thing about that by the way is somebody else in the room overhearing those sounds we have no idea What it's about so like I am getting this ambient sense and also anybody was appearing the same signal unless they were really Watching me and like doing crazy cross-correlation analysis and stuff We have a very hard time figuring out even infinite time in patients what they mean and that's really cool Right this little secret thing that can be giving you an ambient sense of information constantly and I don't think I've ever really built systems like this Today there's been lots of computer human interaction papers on these kinds of systems No one's really made a product like that and part of the reason is because it's too hard to customize you've heard of the tyranny of the marginal user This is Ivan Ben-Dravus piece who is just a brilliant thinker and the frame is if you are a product You're making a product and you want to have as many users as possible It's a typical goal for a product what you do is you go talk to your marginal user the person who used it once and stopped using it And you say what could we have done differently? What would be different and they say I don't know if you were words maybe like fewer features because these are the people who don't like your product And so if you follow this gradient you follow a gradient towards making the lowest common denominator most generic thing that most majority people like The largest amount of people are like okay with but nobody loves and so you dilute everything down to its kind of like default state And this is the path if I were at probably like I had products that have billion users And I go and I pitch this feature that's gonna revolutionize the lives of people who play tabletop role-playing games And we're gonna connect this thing from docs and search and email or do this thing I have to talk to 12 different product managers and convince them to go how many of that new one-day actors will we get from this? How many of these days think about 50,000 like what do we have in this meeting?
This meeting isn't even justified the cost let alone building the future So you get this thing where the what you call the coasting floor of features go Up and up and up within an origin with an origin with a thing when you get more and more generic functionality So no one's ever built this thing just go on goes no one wants that But if you had a tool that was fundamentally built for you that you were using that gave you superpowers you can imagine that the fusion Great you know that how did you know that thing where I had the system where I had to configure different soundscapes and over time It kind of intuitively made sense to me and that allowed me to get a sense without you having to be worried about the fact that Actually getting all this notification information indirectly You can imagine that slowly diffusing out if people could do it themselves someone to the app architecture fundamentally presumes users are passive consumers Can't tweak it can't add a feature that they want it's a people go now. They don't want it Yeah, so this gets at another piece of this thinking like one of the holy moments in this series has been talking with K a lot of McDowell on adopting Fred Turner's framework of like different media environments creating different selves and K's piece a gropius bow on the environment of neural media meaning LLM's and brain computer interfaces Creating a different model of selfhood which is like a persistent vector embedding in latent space And so it's like we see the machine the machine sees us and instead of thinking of ourselves like we did in early internet Network society that you're a node in the network. It's like you're a path through this space I've been thinking about that and how it articulates with the comments by Jeffrey West on urban scaling and the proliferation of Social and economic functions and it's at the extreme end of this and we've started seeing this already with like long-tail economics and Kevin Kelly's Thousand true fans is you're not having to appeal to the tyranny of the marginal user You're having to appeal to the super fan, but then at the very end of that process I don't know if we'll ever reach it But in a sufficiently large society that has some kind of Synoptic view of activity and some kind of exchange within it at that level at planet scale Then the distinct embedding that you take is part of this enormous adaptive reservoir Within that system and so like more and more There's I'm seeing the sort of like the rise of the generalist but the generalist is not a true clade Paraphyletic played in which every generalist is only a generalist because they have taken their own path They followed whatever was most interesting to them at the time and within that landscape of adjacent possibility And so where this seems like it would take us and you're saying this already with like the bespoke software environment that it gets us to a place where Your economic value is about the distinction of the path that you've taken and how little it overlaps with all of these other Embeddings that if it is legible then it can be reproduced at near zero marginal cost And so there's a shift and it's basically like this is related to your comments on content versus art and the ability to challenge the user In the modern industrial society it favors competition for these clearly defined legible positions But once you have enough integration at the top and enough granularity about what's going on down below Then you stop hiring people for job positions And you start hiring people for their capacity to contribute something unique that no one else can and an economy like that Basically favors people who are doing something completely weird like interesting as opposed to more Yeah, yeah the difference that makes a difference like their ability to contribute novel information I had a bit involved that I cut a few weeks ago because I couldn't it was an idea that I maybe ash had it or something that It was really interesting me that was about exactly this case Instead of you fitting yourself to the role that we do today in this modern industrial kind of organizations instead The roles that themselves were to you because it becomes more possible to find and to fit these kinds of hyper structure And in largerization today, it's all about how do you become closer to the ideal of your role not more ideal to you and your superpowers But what this like slot needs this behavior so this might be a behavior that's really good for the company even But it doesn't fit in the slot so let's get rid of it and the way you subscribe as none of these organizations will say Oh, there's a slot right here you fit mostly the slot your right to your extras don't point in and then you try to jump in You don't really fit perfectly and I got something behind you And they go oh just cut that thing off so you fit and it's those are my wings I'm not gonna go why would you're gonna use me for the least interesting part of my ability and yeah I think I think oh I'm allowing qualitative insights at quantitative scale has a huge implications for a lot of processes and emergent coordination And I don't know what the result will be I think you still have a problem of the principal agent problem in organizations If everyone were Kirby's entirely like genetically identical and I'm aligned with the interest of the colony above all else Then maybe a certain kind of behavior, but then you also have a lack of resilience And if you have a principal agent problem of I'm an individual who will work at other companies during my lifetime And have an interest to remain employed and might put in the table of my family That's a different incentive structure than what's good for the company They don't have to be at odds But in some cases they will be at odds and good horse law emerges Fundamentally from the concept that any kind of metric must be a simplification because a map that's just one to the territory isn't useful They can be leveraged by some distillation But that means that there's some distance between them if you have entities in the swarm who are not fully aligned perfectly infinitely with the good of the collective Which only happens in extremely rare cases then there's some incentive to cheat to do the easy thing that improves the metric that not actually improved the underlying thing that matters and that is a fundamental inscapable thing even in a world of LMS unless we were all worker beings I sure hope that God we aren't because that sounds like to me Like will AI make us more the same or what makes us more different as an able to lean into our uniqueness and my hope is it allows To lean into our uniqueness without becoming island into no ourselves It is important for it to be like a pro-social it helps you belong and make connections One of the things that's nice about bad about online communities and good in-person communities in person communities If you're in a small city or whatever and you go and see this person to cut you off in the parking lot at the grocery store You're probably gonna see him again probably a lot actually you shouldn't like get in their face or whatever or someone challenges you in an environment like that I'm gonna live with them You're stuck with let's figure out if you can find some common ground Whereas if you're in this infinite thing where you can cast out any particular community and switch to another one It's quickly as possible if anyone's as many as don't like in that Well, it's for whatever amount and so you never forced to grow and some growth is bad But being stuck in an environment that is homophobic for example in a small environment and that forcing you to not reckon with your sexuality or whatever But some aspects are good about how I realize the indirect effects of my do this kind of thing It's good for me, but harms other people indirectly and I reckon with that I can balance that better and we lose that in online communities where you have infinite Whatever these losers and go to somebody else where people are not gonna make me feel bad for like early Yeah, the ease of exit which takes us back around to this question of how are the policies aware of shared stake I want to articulate that to this thing that you're sitting with and I'm sitting with which is okay in this toy model of a future Environment that we can see sort of industrial mass fabrication becoming mass Customization and that's about the self-actualization and realizing who you are and being the customer in that environment It's not just about the production of a totally distinct embedding within a global economy because we've been saying for this whole conversation Part of it is about maintaining some kind of illegibility the privacy that allows for democracy or the archipelago that allows for rapid evolution or but years ago Listening to Mark Pesci on expanding mind talking about how you're no use to Twitter if you're always on Twitter But it's like you have part of it has to be the ability And every time I've had that assistant emergency boundaries because otherwise you get this infinite poll towards the average and everything Kind of evens out and nothing interesting can happen So like boundaries ideally want to be fluid you want to be fractal you want them to be able to morph and evolve and change So that you're getting intermixing across different levels and across different dimensions But of course you're gonna want some boundaries of things that I'll tell my husband that would never tell anybody else on earth or Things that tell this particular group of people or any that changes over time and I think it's critical for that to be possible So like this scenario of participation in a planet scale economy Indiv-like individual uniqueness I think it's not just about self-discovery It's about self-transformation because if you find out that like your own personal preferences are leading you to converge with other people in the space Then it's like it's impossible to get someone to change their mind if his job depends on it Yeah, actually it flips. It's like your job your economic value depends on you providing constant surprise to that system And it's like part of this overall move to the sort of tower of babble thing that goes on with universal translation The Peter Watts echo proxy a future in which the like I'm only red blind sight Oh, go ahead.
Go ahead. Yeah, the future in which scientists they don't even speak like there is no such thing as language anymore There's the language that each of them speaks and in one sense This is a really promising because it gets back to the fulfillment of what JCR look later in Bob Taylor laid out in 1968 with the computer as a communication device Which is that where the real promise of a computer is not simply in in allowing people to communicate at a distance in geographic space It's like project management as the ability to rotate something out of one person's conceptual framework and into somebody else's It's a matter of like complete dependency on that system Yeah, no one's is universal translator who can figure out how to translate a particular idea from that brain into your brain But you gotta trust it. You gotta trust the translation. It's not trying to like sneak in this little thing on the side or whatever It's enormously powerful I think like imagine all your thoughts are like these emerging thought flowers and they're like embeddings in your particular embedding space that is tied to your Experiences and you and I have some overlaps are embedding space because we both speak English or both US citizens of your whole life I think we have differences because of where we live or life experience We have certain commonalities and so if I want to communicate this thought flower to you I have to figure out what the seed was Well, you can see being see packet of information and shoot it through a really tiny little p-shooter into your brain Which is like synchronous communication or does it normally low bit rate and hope that it takes hold in a similar part of soil in your head And produces a roughly similar thought flower and then what happened for years might be a totally different kind of thing It might bounce right off, but it's enormously expensive to produce these seeds It's enormously lossy But embeddings for example allow you to keep some richness humans can't communicate them But computers can and that allows you to express and find I think about in conversations like the Goldilocks zone if you agree on everything Quickly gets boring you build trust but the object I get it We're similar but then if you disagree on everything or say things that literally don't make sense the other person It's just noise But I don't like this but if it's in the Goldilocks zone where we understand each other but each of us is stretching the other in some way That is the place of mutual like maximal growth of understanding.
How can you find those spots? I've actually did a thing on me with somebody who reached out to me that also writes quite a bit and I had chat TBT to a deep research report I said hi, my name is Alex Kamaraski I'm being with blur both of us published quite a bit Please prepare me an executive summary of the areas we likely agree on disagree on and we have an interesting conversation on and 15 minutes later I give a really nice report about the commonalities and differences. Alex you're more of a sociologist. He's more of a psychologist in terms of the focus I think you agree on these things you probably disagree about this and it's really cool Because we had public reading I get this custom is worth that told me what were the interesting topics to Do with this person to have a really meaty interesting turn into conversation That's cool.
So this is this question of where friction is you know I was just reading Rohit Krishnan the essay he posted on Strange Loop Canada about Silicon Valley's quest to remove friction from our lives He cites Kailas Ganlands essay and I quote he says she discusses how friction is effectively relocated from the digital into the physical world While we move into a simulated economy where friction like gravity doesn't apply This is akin to thinking about a conservation of friction where it's moved from the digital realm to the physical realm or at least our obsession with Reducing friction reduces it in one place but doesn't eliminate it elsewhere The sort of general prompt to you is about where if we're thinking about friction as Kailat says friction isn't the enemy It's information. It tells us where care is needed and where attention should go Then we have a sense of where the friction has been in the kind of system you're talking about where does the friction end up? I think first of all I think meaning comes from class interview roof friction You remove meaning typically so like the like the things that you choose to do despite the fact that they're expensive implicitly You're meaningful. Otherwise, why are you doing them?
And sometimes that's an accidental quirk of the cards of dissidents that you're trying to reconcile it sometimes I care because it's a thing that is a deep infinite goal of mine So I think getting rid of all friction is not a good thing And friction also will pull you down into the catchment basins of like engagement maxing or whatever It's always things that are just like salt fat gossip us versus them stuff It's like the petro basement of like our brain stems and the other point I make is that the physical world is like many orders I'm actually more friction than the virtual world and so a ton of stuff that makes the technology go Oh, wow, we're really good at this. No, we're just playing on really easy mode because it's fully in the bits We're all the bits if you gotta have atoms that move around in certain configurations in the world like physically being transverse that it's Or is mentioned more difficult and you argue that one of the reasons that strong aggregation effects in in virtual communities is one the same Ortepare paradigm, I think really accelerates this but two it's also intrinsic It's just that it runs the clock speed way faster because the friction is so much lower So you see the sucking in of preferential attachment faster and so reducing when you introduce the friction of the real world It makes it harder to be disagree with the 2027 report one of the assumptions They make it about AI and AGI What one of these options they make is oh once I start making robots on demand and it's all over it takes time and effort to get the materials Make them and like that's like whatever That's like a fundamental universe only works a certain clock speed kind of it has a lot of physics that do not exist in the only virtual realm So I think those different ones are very different I imagine if you really think through the implications of the flip the current shift I'm talking about you can get some really weird places Really quickly and go wait a second. Why do retailers exist in this world? If you follow the thread which Bernie I've done for many years over various bottles of wine and long conversations It gets kind of funky and it gets totally impossible for me to reason about bad things will happen in whatever emergency we're talking about Because they always do but like my hope is that we would unlock tons of good things that are sitting there waiting to exist that cannot exist in our Current laws of physics, but it's very hard for me to imagine how does this evolve in five years for ten years?
I honestly don't know maybe advertising I'm sorry about advertising everything actually does do a useful thing which is it helps you identify it There's a thing that you might want to do that it's annoying It's constantly blaring at you and has no real ranking function that is offering to you But if you had a system that said here my preferences here's the thing I care about and you had a bunch of advertisers that were trying to send Stuff it only got through your filter if your filter actually thinks it's useful to you and earnestly thinks it's good for you You okay, I like that. Yeah, I got the same three. I really want to get through me But I think it's a really great option like the thing you care about it's a really good option for it It's cheap as a good quality of use from sites you trust like no It's a pretty good one that's great. I love that and that makes a thing where I as a user get I think that work for me and also advertisers should like it because you make quality products and that you can convincingly explain why they're Quality and you get through and so you can imagine all kinds of weird implications and shifts and what have you and it's dizzying honestly I think about because it's so many things were used to the force of gravity pulls down And now I'm saying oh the force gravity actually pulls that way.
Okay, got it No, everything changes like I let go of the ball and it smashes through the wall Like there's all kinds of implications that change when you change this thing that felt like a fundamental office again It was just a human law that we came up with that was this convenient law the use for the past 30 years, but yeah This shift the center of friction being in the physical world It's not exactly a qualitative shift But it's like we do live in a world where you used to have more in common with your neighbors the geographic quality Then you do with the people in your discord server And now like you're saying the path dependent preferential attachment in that space being so much easier One of the things I'm noticing is like solving for the millennial crisis of I don't find meaning in my work You've got the most meaningful work situation you can imagine LLMs have solved product market fit for you, but there's this issue of not knowing My work is just to continue to pursue this thing that I find maxes relevance realization for me locally I'm not aware of what value it possesses to other people and thinking like an ecologist I'm like maybe this is not a problem because trees don't really understand what they are doing for Funky and animals everyone's just following their own sort of revenue model and then their waste or pollution becomes the input for somebody else And there doesn't have to be a unifying ecological self-awareness in which you know those loops are disclosing themselves If it's an as a break I was talking to somebody years ago and advising them and they were working in a ranking environment Or they were trying to figure out a ranking change it was some junior product manager a company for advice I know that well they said I'm not here for coaching. I'm here for object level advice. How do I get this proposal shipped? Okay, and so it's a bunch of your engineering manager thinking what's the VP think they go wait there is this proposal bad for the Set of creators and they go so I'm not like terrible but distinctly worse than today by some margin and it's not bad for our users It's not bad for us.
Isn't that bad for society the person goes who's to say because you're the one who's saying that we should pull this lever so shouldn't you have an opinion is this gonna be a good thing for society or bad thing and they said listen my goal was to increase Watch time and major told me that I'm a danger of missing expectations the goal for the company is to increase watch time This one crease watch time what do you want me to do? And it was like oh wow yeah, you're right I just personally will get fired out of the system nobody in the system wants to stay That's gonna harm creators or whatever But like the metric does in the short term and you fall into this stuff where I think if you were aware I also want to tell me at one point you'd be a VP by now if you stopped thinking through the implications of your actions and To me that was the most damning statement I thought wow cool. I don't want to be a VP then but I think a world where it's easy to currently to ignore the indirect effects of your actions I think even just being aware of them are natural shame response some wanting to be part of society would help us make slightly more balanced decisions And not saying that you have to change all that much but sometimes you don't even aware that's what I'm saying like therapy or whatever you discover Whereless thing that I do is affecting you and I don't understand it I should do it last because my attention is not to harm you in that way I didn't think through the indirect effects of my actions and once you're aware of them You might still say okay, so do it or you might say oh now them aware of that actually I've got a balance that differently And I hope that we're in a pro-social kind of environment that allows us to be aware of those not as you like frozen by them But just aware so that we don't accidentally steamroll over other people or other things without being aware of it Yeah, that brings us all the way back around to the question I had for you earlier about setting your policy by values and intentions and Twice in one conversation now Charles straws Excelarando and his character manford max using his exo cortex his personal context engine to file patents But not just to file them but to license them to the organizations that he plus his machine believes are gonna have the most Pro-social values and this sort of constantly chasing the question of understanding the consequences of one's actions and how they feed back on the revision of those values You know You know in good place one of the leaders is a character who's frozen by their indirect effects of actions and the most moral person in the universe And they do nothing and that's obviously bad so you want something that's like a learning loop where you are striving to do better over time And you're striving to learn and understand is confirming evidence and be aware that you are just one actor or this larger thing And you get these as me back and you can learn and I think that that is important characteristic You don't want perfection because that would then everything would be totally frozen You want something that can respond to those signals absorb them reflect them That's why I'm saying the example of a thing if I had a I could play right up a value statement I could talk to an LMM for ten minutes and have a value statement that would probably be pretty consistent for most of my life for this period of Life I want to spend high quality time with my kids having deep meaningful interactions with them in the world and the people are alive They're important to us or whatever and I don't want to maximize amount of interesting new ideas I learned that challenge by thinking that expand my circle of empathy that helped me be a more present Proactive member of my community and my family But we really reduce it down to a small set of things a lot of things with high leverage kind of come out of that And so then dooms rolling on LinkedIn because you can't get your Twitter and Facebook fix pretty obviously I don't like my intentions and if there's anything right now you said you didn't want to do this I think it's crazy But it told me a few days later after we become sober from that interaction say Does that actually align with what you intended? I can say yeah, that's it or it says no judgment just okay?
Then it doesn't okay. That's just make sense. We're gonna tweak it so it's like this Yeah, it makes sense and I'm I don't know just very powerful about that But again it has to be aligned that's like you have like intentional tech. I want big tech I want better tech I want tech that aligns with my intentions with my aspirations and my agency and that is not a thing that we fundamentally get today in the Model where the person who owns your data is the one who wrote the code for whatever strange reason and so that's a thing That's a thing that needs to change But if we do I think I think that will be as massively positive force for human flourishing and we can make that happen We just gotta do it.
Yeah, yeah I guess that there's something that's still sticky with me and we've been touching on it throughout this conversation Like the last little piece to work out is and this is related to this idea of like bespoke software on demand and like a lot of That you said in the era of infinite software software becomes an afterthought and so you end up with a kind of like shitty software apocalypse Well, most of it is ephemeral. This is another thing I talked about with Sam It's like there's something about always being in the feedback loop the living in an environment of ephemeral software But you need those long slow beats you need the slower pace layers and you the meaning you need to have a This is why I think software should fade away your data your context your meaning is what should matter and the individual bits of software should be an afterthought By the way, you can't wait for your case All that's a bit software it's from 2004 is actually the only copy on the internet now is on a GIRM Mirror and the basic frame is situated software is software is situated very specific social context like just for me or just for the two of us And every time you built a spreadsheet or customize it That's a piece of situated software into anybody who's not situated in that context. It looks like a piece of shit barely works It's ugly. It's insecure, but the people who use it it's perfect It's exactly what they need and if they didn't feature they will simply add it and if there's an insecurity that will simply fix it And there's something powerful about shitty software from somebody else and gross So he's all for me beautiful.
Well, everyone likes to smell their own is that yeah, exactly I'm not so great I think but it's like I guess the general thrust there is it seems like the fact that the software world is so frictionless It clearly has enormous leverage on the physical world and it's running into these natural constraints But like more and more there's a kind of a femoralization of selfhood in these environments and if all of the different Context are kind of arising and decaying we end up with a self that is not a persistent embedding Or we're not going to be involved in all the way front right as in we've always been that way and now I think we're closer to the the idea people who are at different stages or that Adult mental theory of like self-authoring self-transforming and whatever have different mindsets of themselves as a person Oh, I am this way or do you acknowledge that you can grow and change that your beliefs can more over time? And that's not necessarily a bad thing at the strength I think it's good in a world where if you were aligned with your aspirations and your aspirations will change my aspiration is to be thought Of a cool guy by these people in my school. That's the thing they might care about in high school And later I might say actually know my aspiration is to do things to be proud of the indirect impact of my actions and that be proud of whatever My tombstone and it's impact on humanity. That's the thing that really motivates me and maybe I know maybe when you're later in your life And you have to teach some of those things your goal is to just be a thoughtful member of your community and your family that helps them feel Connected and hold whatever it could change over time And that's okay as long as it aligns with your intentions as opposed to your like a stews roll or whatever Yeah, it's just like when I speak to Eric Davis and he talks about there's a difference between plasticity and flexibility In these systems like the consistency or I like thinking about where in a system should it be squishy where in a system Should it be hard not to go in Brander and squishy computer?
Yeah, right like where do you want it to be adaptable? And it seems like the pressure is not toward the sort of persistent Values nucleus that the modern world believed in the pressure is toward being responsive that therefore the threat is Okay, number three in the conversation Charles Ross's glass house where he says the digitization of the human being creates a system Which oh we've defeated death But now you have this existential horror of you never know if you've been hacked and we're already there It's like fifth generation warfare like you never really know that your values are your own Yeah, yeah Intelligence or system that has capacity and order magic or to beyond you you don't even know if you're being played like you can perceive it If the scramblers can think through a magic fashion new or have Computation three words rather than you then you realize you can't perceive as if I many years ago and they're talking about yeah With self driving cars I thought how are they possibly gonna model the intentions of all the humans and where they might go whether plans are in the engine Here goes oh no, we run this at thousand times fashion humans think and so we don't need to we simply see did the person step into The street and if so we respond immediately don't have to have a theory of mind of each pedestrian We just watch what they do We're responding so much faster than humans think and that really stuck with me like this notion of different pace layers are like almost mutually incomprehensible to each other I can't even comprehend three Experiences the world because of the totally different pace to me Have you seen this old German stop motion from the early 2000s called Das Rod? You see this you should look it up. It's a it's a little stop motion It's about these rocks that are alive, but they move there's like a stack of rocks and they move many words You're bad to slow than humans do and it just shows human civilization and them next to each other and they can't make sense of each other It's well done.
It's cute and it's kind of mind-expanding. Yeah, so I don't know it seems like we don't have much choice there Yeah, dude. This was fun. It took forever to get you on the show, but it's literally a thousand times But yeah, half of them were mine.
I think it's like three quarters of them were mine But okay always a pleasure just imparting if you have one piece of advice for people about living in the farthest You can see into the future. What is the one context independent thing that you suggest people allocate their attention to? I think using LMS to think more not to think less But I think more because now you can think 10x cheaper than you could before and owning your contacts owning and maintaining that context of like your Memories or whatever being on a turf that you fully know is aligned with your incentives I think is really important because before if you had all that information in one place was a possible infrastructure scale to make sense of it with all of them to think that it's imperative that's on your turf in a way that Wasn't really as quite as imperative before if we live in a world with demons we might as well make them our demons. Yeah, yeah Awesome dude.
Take care Thanks again for listening if you enjoyed this episode and you want to learn more links to Alex's writing along with tons of related episodes And essays of my own can be found in the links below there You will also find links to our discord server information about membership in the humans on the online community And how you can get all of that stuff by making recurring tax deductible contributions at every dot org Humans on the loop is a listener supported project and it exists entirely thanks to the generosity of people like you next week We have a conversation on trust in the digital age with one of my many Australian friends the wonderful Nate Kinch followed by conversations with legendary Biologists Stuart Kaufman Tyson Yunkaporta and many more until then take care and remember Attention is our greatest natural resource Although of course that might mean our attention is someone else's resource in which case another good reason to be mindful