エピソードの紹介
ken
Hello and welcome to another episode of London Tech Talk.
I'm Ken, your host, and this episode is brought to you by me and Kaz.
We both are originally from Japan and we are working as a software engineer in London.
Kaz is unfortunately busy today and he cannot make it, so I'm your host today and I invited a guest.
So let me introduce him first. Hello Sudip. Welcome to our podcast.
Sudip Ghimire
I'm Ken. Thank you very much, Ken, for giving me yet another opportunity to begin this talk.
This time, the talk will actually be Tech Talk, I think, right?
ken
Yeah, exactly. Definitely. This is actually your third episode to join us,
so I'm sure some of our listeners already know about you, but I also know that new listeners start listening from the latest episodes.
So can you give me a quick intro, please, to our new listeners?
Sudip Ghimire
Yeah, definitely. Hi everyone. So I'm Sudip, currently working as a remote software developer from Nepal, by the way.
So let's get started, maybe, right?
ken
Yeah, sounds great. Okay.
So today, we're going to try a new recording format.
We usually invite guests and talk about their career history, sharing their inspiring stories and sharing what they are passionate about to our listeners.
And for Sudip, we invited him for the first episode, Sudip talk about your amazing career history.
And then the second episode was a we talk about your beautiful country, Nepal.
But for today, each of us, me and Sudip will bring one technology topic and share for each other.
And at this point, I'm not sure what Sudip will bring up.
I've heard that this is going to be like AI assistant colleague, but I'm not sure what kind of it is exactly.
And I also did not share what I'm going to talk about with him beforehand.
So surprise, surprise. It's like our usual one-on-one catch up where we enjoy discussing about recent tech trends and sharing exciting news.
But we're doing in the recording. So I call it like a public one-on-one style episode. Do you like naming?
Sudip Ghimire
Well, yeah, yeah.
ken
Cool, cool.
Sudip Ghimire
Let's start first.
ken
Yeah, let's do that.
So actually, I'll go first because, you know, I'm facilitator.
So I feel the responsibility to bring something exciting for us.
Okay. So as you know, I'm database geek. I love database.
So the first topic I want to bring in is about database.
So today I'm diving into the world of serverless databases, specifically one that's been making waves, Cloudflare D1.
Have you ever heard of that?
Sudip Ghimire
Yeah, I know that, right.
ken
Oh, cool. Have you ever used that?
Sudip Ghimire
Oh, well, to be frank, I haven't used it.
ken
That's cool. Yeah.
Yes, I'm not. Maybe.
That's right.
Sudip Ghimire
Database guy, right?
ken
Yeah, that's my time. So no worries.
So yeah, I've been tinkering with it.
And I'm genuinely excited about its potential so that I want to share why I'm so excited about what the Cloudflare D1 is and how I use it in my hobby project.
And actually, it's great that if you have never used it, because it challenges me to explain a difficult, hard topic in an easier way so that you can understand.
So yeah, it's like a challenge for me to explain how great Cloudflare D1 is.
Before I start topic, one quick question to you.
What kind of databases do you use as a backend engineer, like SQLite, Postgres, MySQL or MongoDB?
Sudip Ghimire
Yeah, well, for me, I generally prefer, you know, generally relational databases rather than document based, right?
So I have used a lot of relational databases, specifically Postgres.
It's my primary database and I choose it among other because this is one of the fastest and open source database tool that's available, right?
But in my organizations, I'm also using MySQL as well as SQLite, but in the small scale as well as for mobile applications.
So that's all I do, right?
So I have no SQL, but to be frank, I really prefer, I mean, you know, relational databases or document based.
What about you, Ko?
ken
Well, I use a lot, I don't know, I'm not backend engineer, I'm SRE, so I operate them rather than writing SQL query.
And that's why I'm very excited about like serverless databases because it could potentially reduce the burden for operating from our side.
So yeah, before I start, I want to make sure that I'm not behind Cloudflare.
I did not gain any sponsorship from this company.
And although I use Cloudflare at my workplace, this is my individual thought and opinions as usual, nothing to do with what I'm doing at workplace.
Cloudflare D1の基本
ken
So yeah, for listeners and you who might not be familiar with Cloudflare D1, so I'll start with a very basic topic.
So what is Cloudflare DUI and when did it pop onto the scene and like what's the core idea behind it?
So I, if I remember correctly, it was announced back in May 2021.
So almost like three years ago and hit GA, general availability around April last year.
So it's pretty new product.
And after the first announcement in 2022, they seem to gradually roll out secret customers.
But publicly it became open beta, they call it, in November 2022.
So this is a time when most of the developers see Cloudflare DUI in the wild.
And their product line, their marketing strategy, their pitches was like simple and instant serverless databases.
So the core idea is to let developers like you, Sareb, to use SQL, a language most of us know, directly from Cloudflare workers, which is a computation layer running right at the edge.
So it's like aiming to remove the headache of managing databases, installing binaries, setting up configuration.
And actually it's built on top of SQLite, so RDBMS, your favorite one.
So yeah, that's an interesting choice, right?
So I remember you said you use Postgres most of the time and MySQL.
When you used that last time in production application, how did you deploy that?
Did you deploy Postgres into VM boxes like fly.io or did you use any serverless databases?
Sudip Ghimire
Yeah, well, to be frank, I mostly prefer using SQL databases with AWS's RDS, right?
Because it's so much scalable for me.
ken
I love it too, yeah.
Sudip Ghimire
Yeah, right.
So I prefer that RDS for that.
And maybe for some smaller scale applications, I use just containers because, you know, Docker is everywhere nowadays.
ken
It's a pretty common industry standard, isn't it?
Sudip Ghimire
Yeah, yeah.
That's now a really pretty common standard.
Talking about D1, so what do you think about D1's architecture regarding SQLite?
I mean, aren't you more surprised that it is not using larger scale databases, something like Postgres?
And they are preferring really light databases like SQLite, right?
So there might be something.
ken
Actually, that was a great question at the great timing.
I'm very surprised that you are, you know, asking like a second or third host now.
Because I was going to explain about the Cloudflare D1 architecture.
So yeah, let me explain how Cloudflare D1 is built.
And as I cannot see their source code, this is my understanding from doing the research by reading the official docs and the article.
And actually to explain D1, the architectures, we actually need to introduce another Cloudflare product for us.
It is called DO, Durable Objects.
Also, yeah, it's known as DO.
Yeah, capital letter DO.
Yeah, so let me explain DO first.
So DO provides, it's like a fascinating building block for stateful server applications.
And think of DO instance as a single threaded piece of computations with its own durable strings attached, which is SQLite.
So DO is the combination of like computation with SQLite for each instances.
And DO was actually was announced before D1.
It was as far as I remember correctly, it's around 2020, two years before Cloudflare D1 was announced.
And, you know, that the key idea for DO is like the Cloudflare's, as you know, it's CDN, right?
Network routes all requests to a given.
And it, as for DO, Cloudflare network routes all requests for a given unique durable object ID, UUID to the same instance.
So you can have a global unique ID for each DO instance, which means one process single threaded talking to SQLite.
And no matter where the requests originate from.
And so that the use case, I'm going to talk about a use case, but this is single threaded architecture nature for specific instances.
Like crucial for maintaining strong consistency.
So, well, it doesn't scale for write and read, but, you know, every request with the same UUID go to the same instance.
So it's great for the requirement when you need a strong consistency, for example.
And so like when the DO was announced, they marketed as a great tool set for building like collaborative applications.
I remember you say you work like collaborative application using CRDT, right?
Sudip Ghimire
Yeah, I was about to talk about it because there were so many exceptions regarding durable objects from Cloudflare.
D1データベースの基本
Sudip Ghimire
And I, in fact, went through those error messages and eventually I just went to the DO and I got to know something about it.
But to be frank, I was not working on that core level.
So I just skipped, you know.
ken
If you remember, you can share more about that.
Yeah.
Sudip Ghimire
Yeah, definitely.
At some point, maybe.
Right.
So when I was using Yjs for collaborative application, specifically for collaborative document application.
At that point, it was mostly used in WebSocket at that time.
So I got to know about it.
But as I previously said, I'm not so much familiar with it, by the way.
ken
But you know the concept. That's great.
Yeah.
So you already know about basically what DO is, right?
So what is the relationship with DO and D1?
So each D1 database instance is fundamentally powered by DO instance.
Sudip Ghimire
Oh, that's great.
Yeah.
ken
Yeah.
Sudip Ghimire
So now it makes sense.
Yeah.
ken
DO is basically the building block.
And each DO is just a single-threaded SQLite computation block.
And then they build like a streaming layer, replication layer and distribution layer on top of that.
And it becomes D1.
Sudip Ghimire
Oh, wow.
That's an amazing architecture.
Yeah.
ken
Definitely.
Yeah.
Sudip Ghimire
Right.
ken
So but stepping back, it's actually simple.
They already have DO as a building block.
And they build their own streaming and replication distributing part on top of that.
It becomes globally distributed D1.
This is basically what D1 is.
And I want to quickly touch on the distribution part as well.
And so as far as I've read a couple of research and article under the hood.
So Cloudflare uses their internal system called SRS, which is Storage Relay Service.
I cannot see the source code, but which appeared to be inspired by the technology called open source tool called Lightstream.
Sudip Ghimire
Okay, I see.
ken
Yeah, Lightstream, which is actually one of my favorite technology.
Well, I didn't use it at production.
But by looking at source code documentation, it looks like very clever open source tooling.
And this is developed by an engineer at fly.io, by the way.
And yeah, this is basically the streaming replication tool for SQLite.
Streaming replication.
Yes.
And how it works is pretty smart.
Taking snapshot and using the write ahead lock as streaming to continuously capture changes.
And it uses any object storage as a snapshot, like AWS3 or GCP bucket or whatever.
ken
And Cloudflare said in one of the blog article that they built SRS, which is the internal system inspired by Lightstream.
But they built it from scratch.
And to handle things like streaming the write ahead lock from the primary DO's SQLite writers
to their Cloudflare R2, their object storage for backups and time travel.
And then they also use SRS for propagating the changes to other read replicas, which is globally available.
And so by mixing SRS with DO, they achieve a global distributed DO databases platform.
This is my understanding.
ストレージリレイサービスの詳細
Sudip Ghimire
Oh, I see.
That's great, in fact.
And now I'm more thinking about how they are handling, I mean, those pipelines.
I mean, multiple origins and multiple databases, how they're working on it.
That's amazing technology, in fact.
So let's see, right?
ken
Yeah, the pipeline is the thing I'm going to introduce next.
Why you can ask so many good questions at the right time?
So yeah, so as it says, so they used SRS for pipeline, so that this is actually the recent announcement in last month, this April 2025, that it's support for the global read replications with sequential consistency.
So yeah, I'm not sure how much you're familiar with the database consistency, but SQL consistency is called a read your write consistency.
Right, and okay, let me, you can chime in if you have any idea, but the read your write consistency is, let's say you're building the SNS like Twitter, and you post your messages, right?
Yeah, right.
And you go to your own feed, or you reload the browsers.
If you can see your latest messages, it means Twitter background is sequential consistency, because you can read your own write.
Let's say if you reload browsers, but if you cannot see your own feed, which you posted one second ago, which means your database does not support sequential consistency, the consistency is below that.
So it depends on the like, but sometimes after like waiting 30 seconds or a few minutes, you keep reloading.
データベースの整合性
ken
And if you see your post after like, let's say one minute, it's called eventual consistency.
So after waiting for a bit, you know, the consistency is like you see what you've wrote.
And if you can see your own writes instantly, it means sequential consistency is guaranteed.
It's a bit of databases.
Yeah, it's.
So if you understand consistency, you can pick the right databases for your requirement, right?
You don't have to pick strong consistency for the application that doesn't require strong consistency.
Because basically, if you need to guarantee the strong consistency, databases in theory is slower than eventual consistency one.
So this is how it works.
Sudip Ghimire
Okay, I see.
So I have a quick question that might be or might not be relevant.
But for example, we want more sequential consistency, for example, in case of some, for example, stock market data, right?
Because it needs sequential consistency.
So at that point, is it, I mean, better to use sequential databases or generic databases tools like, for example, Postgres or maybe even D1, write path to D1?
So what do you think?
ken
So your question is when you build like an application of stock markets, viewing data, which consistency you need, right?
Sudip Ghimire
Right.
ken
Actually, if even if you pick Postgres or MySQL for some databases, it's configurable.
Well, so you can, in the, I don't exactly remember which configuration level you can, but many industry, you know, proved big names databases support one or two or three different isolation level.
So you can change it.
Sudip Ghimire
Right.
ken
So, yeah, if you're building for the stock market app, and if you need like a strong consistency, please check the default value for the databases.
And if it's not enough, consider tuning up to the consistency level you need it.
Sudip Ghimire
Okay.
Thank you very much.
I understand.
ken
No worries.
And the strongest consistency that is supported from your databases is not what you need.
You have to switch to another databases.
Oh, I see.
Yeah.
Sudip Ghimire
That's again, another pain maybe at some point, right?
If you do not use it, I mean, at the right time.
ken
Yeah.
But as long as you're using Postgres, it should be fine for most of the cases.
But well, I'm not sure about the stock market app requirement in details.
I cannot say anything, but yeah, this is starting point.
Sudip Ghimire
Yeah, right.
I was just, I mean, more, you know, concerned about whether should I use specific sequential databases or use regular databases and do more and more indexing on time values and make the database grow larger, right?
ken
Right.
Yeah.
Definitely.
This is just a trade-off.
You first understand your business requirement and then you understand what kind of consistent model is required.
And then you just pick the right databases for your application.
This is, yeah, trade-off as usual in decision making, right?
Sudip Ghimire
Yeah, right.
D1の特性と制約
Sudip Ghimire
Maybe we're a bit, you know, deviated from our topic, but maybe I think it's your time to talk about D1 again, maybe.
ken
Wow, you're so amazing.
You're now facilitating the podcast.
Thank you.
I often talk about a relevant topic, but okay.
Going back to D1.
So as I told you, they build the O and so SRS, what was that?
Server Relay Service or whatever.
And so there's one limitation.
So read can scale, but write doesn't scale.
You know, that does make sense because write primarily is basically one DO instance.
So if you want to scale up the write, it should be limited by how fast, you know, each single DO instance can handle.
So this is the, how can I, it's like this advantage or limitation that you should consider if you're talking to the D1.
If your requirement is read heavy, and if you are like an indie programmer, you don't have DevOps, or you're working in a very, very small startup, you don't have SRE within that serverless databases and read scale.
D1 could be in an options, but write doesn't scale.
You have to consider to like sharing within D1, which doesn't sound like a good idea.
So yeah, write scale is something you should be aware of.
Sudip Ghimire
Yeah, definitely.
So I'm just thinking about some scenarios, something like maybe we might have some, for example, database replication, maybe, right?
And even for those replicated databases, even if we have, for example, a whole backup of another, the same database in another maybe availability region at the time, write cannot be, I mean, scaled up, right?
Cloudflare D1の機能と利点
Sudip Ghimire
Because it should go to some pipeline and it should eventually be written to all of those databases.
But while writing, it should be first changed into one specific database.
So I think that makes sense.
So this is what I mean, yeah, right?
ken
You get the point.
Definitely.
Yes.
You're already ready to use Cloudflare D1 for your next project.
But yes, that's the point.
Yeah, you get it.
But stepping back a little bit, the great thing about serverless database like D1, it comes with replications or the backup mechanism in places.
So for example, D1 cases, it has a feature called time travel.
And time travel is basically, they keep the snapshot.
As I told you, they keep the snapshot into their object storage.
And so for up to, it depends on the pricing plan, but for the free plan, up to the last 30 days or so, they keep the snapshot in the object storage.
So then you can easily went back to specific versions for free.
It's pretty good.
If you install the Postgres by your own cell using Docker container.
And if you want to do such kind of like backup, you have to install a few more tool by yourself, right?
Sudip Ghimire
Yeah, that's right.
That's really, I mean, you know, the hardest point about the local development.
Because we need to use another tool for backing up.
And sometimes the problem is whenever we just take a backup and try to restore it.
Everything gets messy.
I mean, yeah, most of the times if we use some sort of, you know, kind of auto increment values, it might collide, right?
Because there might be existing data.
So definitely another tool might also not, you know, fulfill the requirement that we are maybe requiring.
ken
Yeah, yeah.
I mean, backing up data for your database is a very classic problem.
So there are so many hadn't tested tooling, even for Postgres and MySQL.
But as long as you're installing your binary version, you have to set up by yourself.
So it's not ideal for like prototyping or your V1, you know, applications as an indie programmer, right?
So this is why like serverless databases like that, you know, shine in such use cases.
So lastly, I wanted to share how I use D1 in my use case.
So I have my own personal website.
I know you also have it.
But in my case, I use D1 as a background to store all of my personal blog posts.
And I store all of post metadata in Cloudflare D1 and serve it through the remix.
It's now React root of V7 and application running on Cloudflare pages.
So yeah, as Cloudflare market, I use, you know, the recommended pattern, you know,
Cloudflare D1 as a stage and Cloudflare pages will work as a computations.
And what I love, what I like about D1 is the integration
with workers feels very incredibly natural.
You bind your database from your configurations.
And you're writing standard SQL queries that are comfortable with SQLite.
And the performance potential is very massive.
Like, you know, having read scale and read scale is, you know,
important for my, you know, Brooks ABC's book.
High write workload is not necessary, but, you know, I needed like read scaling.
And so just picking serverless D1 databases for my blog cases was really,
you know, the great option available in the market right now.
But I feel that it's a little bit too early for massive scale dataset
for your application as you already feel.
It handles really well, but, you know, the tooling around still is being developed.
Like global read replication is still in early phase.
So there's no good schema migration tooling, you know, the end metrics.
And debugging process is pretty basic compared to other production solution
like RDS, you love it, and the GCP.
So, yeah, I see potential here, but as of today, it's good for startup or
Indie softwares, but not something enterprise company want to use it.
That's my impression.
Yeah.
Sudip Ghimire
Yeah, yeah, that's right.
Because as you have previously said, if I remember it correctly,
it was first introduced in the year 2021, right?
So it's maturing.
I mean, it might take some time for larger products to maybe, you know,
accept it or maybe trust it, right?
But eventually it will because I could just see those things.
I mean, they are just, you know,
evolutionizing these database related events.
So it will eventually maybe or may not be, I'm not sure,
Cloudflare D1の概要
Sudip Ghimire
but it will eventually replace current solutions.
ken
Yeah, maybe.
Sudip Ghimire
If not, it will definitely try to, right?
ken
Definitely, yes.
And yeah, the last call is, but to be honest,
I'm excited not only with Cloudflare D1,
but I'm usually excited about all other serverless databases
available on the market.
And especially it's for SQLite,
which is one of my recent favorite topics.
So another product I'm recently interested in
recently is called Tulso.
I don't know how to pronounce it.
I think it's from Finnish.
Sudip Ghimire
Oh, okay.
ken
Finland, yeah.
Which seems to be another exciting product
for the SQLite distributed databases.
So at this point, I'm still beginner to talk about it.
So I'm not going to talk about it today,
but I'm tinkering with it now.
So maybe next time.
Sudip Ghimire
Yeah, maybe we can have another episode for that.
Maybe it might be more interesting.
ken
Cool, yeah, yeah.
Yeah, that was me, the Cloudflare D1
that I wanted to talk about.
If you don't have any follow-up question,
now next is your turn.
Sudip Ghimire
Yeah, maybe I might not have,
but I'm still thinking about the SQLite
and those indexing things.
But because as I am not so much familiar
with this topic,
I might not be ready for the question, right?
Maybe in the next episode,
we might talk about it
if any topic that is relevant to D1 comes.
ken
That's great.
AI支援コーディングの進化
Sudip Ghimire
Is there anything else you want to talk about D1?
ken
No, just go ahead.
Sudip Ghimire
Yeah, well, so it's my turn then, right?
ken
Sounds good.
Sudip Ghimire
Yeah, okay.
ken
What's your topic?
Sudip Ghimire
Well, I'll be talking about
really interesting topic, in fact,
about the topic.
Something that's transforming
how we write a code.
But nowadays, we can just think something like
something that is transforming
how we see the world, in fact,
because it's not just impacting coding,
but everything, right?
So we'll be talking about,
I mean, especially I will be talking about
artificial intelligence in coding.
ken
Hmm, whatever topic.
Sudip Ghimire
So, well, to be frank,
AI is gaining a lot of momentum, right?
Not just in coding assistant,
but in creativity also.
I mean, you could just generate
images of your imagination with just,
I mean, within a minute, right?
Just by describing the situation, for example.
We even do not know how, for example,
a mermaid looks like, for example.
And if I just describe
how a mermaid would look like,
for example, it might have,
I mean, body of a human and a tail of a fish.
And it will generate the image
so that you could just visualize it.
So previously, it would take hours,
even days for an artist to,
I mean, imagine and sketch it out, right?
As I'm also, I mean,
sometimes I do sketches, right?
So I know how long it takes
to just sketch something.
And you can just see,
AI is just doing it within Snap, right?
So that really is amazing.
I mean, so,
just before talking about AI assistant coding,
we need to know how it all started, right?
I mean, it came since generations.
I mean, before starting with this
perceptron or whatever,
and the aliens.
So I still remember those time
when we had to read many books
or maybe many references to solve problems.
Or for example, for SQL,
so we even needed to look through
hundreds of blocks
to just optimize one SQL query, right?
And now, we can just think of that.
It was, I mean, the time
when we had to look those books
or references or sometimes catalogs,
if we go really down to the past.
So that time was completely made obsolete
by tools like Google and Stack Overflow, right?
ken
Yeah, like search engine.
Sudip Ghimire
And looking for the resource for weeks and months
was just cut down to some few hours or minutes.
And now, you can just see,
I mean, we could,
I mean, we used to bang our head
thousands of times
and not finding a perfect solution.
And now, you can just see,
they're also, again, infected
because AI is being superseded.
I mean, they are being superseded by AI.
I mean, not just finding a solution within Snap, right?
So you can see those
whole software development life cycles.
For example, even my current organization,
they're just trying to, I mean,
reduce the human resource
AIの影響と進化
Sudip Ghimire
just for the sake of productivity.
For example, reviewing the code nowadays,
it's so hectic.
Even when I had to review the code,
I think you'll understand my problem, right?
Because another developer is going to write the code
in some other fashion than I do.
And I have to review that.
And I even do not know
whether it is perfectly valid or not.
And there might be some constraints
Sudip Ghimire
maybe regarding performance
and maybe multiple objects look up
or whatever, right?
Sometimes.
Maybe if we are using tools like Python,
we have to be aware of global interpreter lock, right?
So there might be a lot of issues.
And now AI will just analyze
whole code within a second
and it will even give you the perfect solution.
I mean, it will even create statistics for you, right?
It's really transforming
the software development life cycle.
Yeah.
ken
So I want to chime in here, actually.
What is your honest impression?
Are you scared or excited?
Or do you have a mixed feeling?
Sudip Ghimire
Oh, yeah.
Well, to be frank, I have a mixed feeling
because it's helping a lot in many ways, in fact, right?
So it's not yet mature to be scared, to be frank, really.
Because currently we can just see those GPT
or whatever the LLM models, right?
Currently they are just transformers.
It just transforms the user input
into the different layers.
I will be talking about those layers lately.
But let me just give you a quick introduction
to the phase where the current AI is.
Because the AI currently is not the generic AI
or the general AI
because it's specific to a few things.
So it's not currently multimodal
even though Google and OpenAI
is currently working on multimodal
or multimodal inputs and outputs.
It's still in the phase of development,
but yet it is a state of art technology, right?
So I will be talking about both potentials
as well as some issues with the AI
and its maybe future.
Because there are some things
I'm concerned about,
but not about the, maybe, let me just say,
it's not about the effect of AI on humans,
but the effect of AI on the AI itself.
So that might be interesting, right?
ken
So you're talking about how each AI's output
interacts with other agents' inputs.
Sudip Ghimire
Exactly, yeah, that's right.
ken
And your concern is more like
it becomes a vicious cycle,
like about the feedback flow.
Oh, interesting, tell me more.
Sudip Ghimire
Right, I mean, currently I know
the AI is not that much mature enough.
I mean, everyone knows, right?
The room wasn't built in a day, right?
So it all started with smaller, I mean, tools
like maybe you might also have used
JetBrains or, I mean, IntelliSense, right?
That time, previously when I used
Turbo C++, I needed to remember
each and every methods of those classes
and the parameters and doing all of those things
like operator overloading myself,
but it was eventually replaced
by those IntelliSense.
And now if we disable the IntelliSense
and if we are, for example,
given a basic text editor application,
we cannot code, right?
So it started from there,
but we, I mean, almost forget
about those class methods
and whatever parameters to use, right?
But with the help of AI,
now, you know, you can just think like
we do not need to think
about the problem itself.
We can just give some instruction
to the chat agent
and it will provide us the solution
which might be degrading
the performance indirectly.
For example, it might affect
our problem solving
or maybe even critical thinking, right?
ken
Right, right. I totally get that.
And one example I want to share with you
is like, I was using the agent mode
for writing my some, you know,
hobby project for my indie project.
And then I needed to get some data
from API endpoints.
And, you know, I needed to pass
the bearer token for that.
And then I store my credential
in one password.
And then it could get this credential
from one password using the CLI,
the one password developer, which is, yeah.
And so that, you know,
I try not to expose my secret.
And then agents at some point realized
that they needed to talk to the API
by using the token.
AIの支援とセキュリティリスク
ken
And then they, you know,
used the credentials.
And then they put that into the local file,
which is OK.
Which the local file and m file
was getting ignored.
And then they started, you know,
using the credential in the command line
without hiding.
So that my credentials started appearing
in my local file and my, you know,
bash history, which is basically
not great security, you know, practice.
Sudip Ghimire
Yeah, that's right.
ken
And, you know,
it was agent more, so it didn't stop.
It just keep posting my credential
in my local bash history, which is fine,
you know, but still.
And if I told agent in the first place
that, OK, you should do it,
you should achieve it.
And also don't forget about security risk,
then I'm pretty sure agents
will not handle like that.
But if I don't tell agents
about any edge cases or potential risk,
what kind of performance issues
or security issue you should talk,
you know, think about beforehand,
you know, agent just, you know,
started, you know, playing around.
And even for the performance as well,
I'm tinkling with databases performance.
So, for example, I asked them to,
you know, agent to come up
with any schema migrations
or potential SQL query.
And they come up with something that works,
but it scans all the rows
without using index.
And then I need to ask them
to come up with potential
better performances by using indexes.
And then they can answer
like a great index
that I couldn't come up with.
But I needed to ask, right?
Someone like human have to walk
like a gatekeeper or violator to violate.
And after my experience,
I realized that human actually can,
well, might not be perfect,
but they have so many high level thinking.
So, okay, maybe we should think about from security.
We should think about from performance.
We should think about maintainability.
And so I realized this.
I'm trying to demystify the gap
between human and AI.
But this is one experience
I wanted to share with you.
AIツールの過信とその影響
ken
And do you have any take on that?
Sudip Ghimire
Yeah, right. That's right.
So I think the problem
you're currently talking about
is basically being, you know, solved.
I'm not sure whether it is being solved or not,
but the concern is being raised.
And they are working on some newer technologies
like self-attention and perceivers.
ken
Oh, interesting. Tell me more.
Sudip Ghimire
Yeah. So maybe at some point of time,
but I'm not sure until then.
So before I start talking about it,
I just wanted to, I mean,
continue the challenges
regarding the developers' critical thinking
and problem solving, right?
ken
Yeah, go ahead.
Sudip Ghimire
So I'll get back into this topic
just after this one, okay?
So because, you know, right?
I mean, AI, it's really putting our lives easier.
But the problem or the concern currently is,
for example, if we overuse the tools
like maybe copilot, right?
So it could eventually degrade our problem
solving capabilities.
It's inevitable. We know that.
And at some point,
we will completely rely on these agents, right?
And only the concern of me,
I think you have already highlighted it before.
So the scenario is something like
if all of us completely depend on it
to generate the code,
at some point,
what happens is like
they will get fed into
the code that was previously generated by themselves, right?
At that point, they will be exhausted.
I mean, there will be no more,
I mean, newer feeds,
I mean, inputs fed to those models.
And eventually one model will get the output from another model.
And if, for example,
something wrong is generated by one AI model,
it is going to be fed into all of them.
And at the end, right?
The butterfly effect is inevitable, right?
It's inevitable.
And everything will be down.
And at that point,
if we are, I mean,
degrading our performance like this
just by relying on AI agents,
then maybe in the near future,
there might not be someone who could solve this problem.
And we have to start from scratch again.
And it might be too late.
But I'm not sure about that
because I was just about to talk about
those technologies like self-attention and perceivers.
自己注意メカニズムの重要性
Sudip Ghimire
They are currently working on those.
So self-attention is another layer inside the neural network
that will decide whether
they are working is right or not.
For example, currently,
they are just giving us their view
or for example,
the view they have been trained from, right?
So currently they are just generating from the data they are fed.
But when self-attention comes,
it will be something like, for example,
I still remember one
issue with the current AI.
For example, someone was trying to steal a bank
and they were just trying to know the loophole of AI, right?
So first they just asked the question,
so how can I steal or maybe, for example, rob the bank?
And it just said,
no,I cannot just provide you with the instruction to rob the bank
because it's illegal and immoral.
But they started, I mean,
tweaking the AI and made them confused.
And they just started saying something like,
OK, so someone in the bank is trying to access the database.
And for example, the thief is having some issues
with accessing the database,
how the database might have been configured.
And it started to answer the question.
Maybe this might be the issue.
Maybe this might be the loophole.
And they all, I mean, we can just think of it, right?
So it really does not know where the conversation is going on.
So now what they are trying to do is like
add a new layer inside those neural networks
that have self-attention and perceivers
so that they could know what they are talking about
and they could perceive
what this conversation might end up to.
So I think the problem of the self-existing
and giving the wrong data might get better.
I'm not sure, but let's hope.
The companies are working on it
because the popular transformer models
like Activity or Cloud or maybe Lama,
they're all working on these perceivers
and self-attention layers.
ken
That's cool.
Yeah, I didn't know about those keywords,
but I see some LLM already started playing around it.
So for example, I asked some copyright-related issues
and they answered me like,
I'm sorry, I cannot help with that
because it's protected by copyright or whatever.
And so, yeah, a self-attention to perceive layers
sounds like a way to go.
Definitely, yes.
They need some self-verification step for sure.
It's not only for copyrights,
but like the example you share,
like stealing money from the bank,
definitely, yes, you need to fix it.
And that's cool, exciting.
Sudip Ghimire
Yeah, well, the new architecture will be something like
the input layer will be something like
the data we just regularly pass,
which might be single model or maybe multi-model
like we can just pass text or image or video.
There are a lot of things, right?
So GPT models or maybe Gemini also
already supports multi-model,
especially they can accept text, audio and image,
maybe video also, but I'm not sure about video.
So these are the inputs which will be embedded, right?
I mean, the coding embedments will be there.
And after that, there will be a positional inputting.
That's amazing thing.
So positional inputting is something like,
for example, if I say a sentence,
what do I do?
So there are two dos over there, right?
So what do I do?
So the do in different position
brings different definition and different concepts.
Or for example, it has different use case, right?
So first do is for the task, right?
And second do is for the action, right?
So these things are handled by those
positional encoding thing.
So those positional encoding gives different bias
for the same word.
For example, the word is the same,
but it will have different bias
because of which we are currently getting
so much precise answer on those.
For example, even if we just ask some wrong question
or maybe question with wrong grammatical structure,
it still gets the right question, right?
Because of this.
So it currently has positional encoding.
But after positional encoding,
we now need this self-attention block.
And after that, the feedforward layer will come,
which will eventually feed to the neural network.
And then they will start working on Perceiver.
So Perceiver comes at this point
because the output is already generated.
AIアシスタンスの進化
Sudip Ghimire
But they currently do not know
whether the output is good or bad.
At this point, right?
So we might not be...
I mean, it's not the time to be more concerned
about whether the artificial intelligence
will destroy the whole world or not.
Because they are not yet capable
of doing a lot of things.
They are just currently responding
to the questions that we are doing.
I mean, it requires some time
to bring general AI
rather than this specific AI tool, for example.
I mean, I might be deviating from AI
as a coding assistant, but I think, right?
ken
No worries. I love it.
Sudip Ghimire
It's more...I mean, I'm more curious about this also
because the evolution of general AI,
I mean, it should be taught in the right time, right?
So the time still comes.
For example, I'm still dreaming of those,
I mean, general AIs like maybe some C-3PO from Star Wars
or maybe TARS from interstellar movie, right?
So this kind of AI, there's a long way to go until this.
But until we get into that point,
we might need something like Perceivers,
which will eventually be a good tool
rather than, I mean, just giving some suggestions
in, for example, our daily software development life cycle, right?
So before ChatsDB or maybe even GitHub Copilot,
I used to do some AI tools like Kite,
or there was another one.
I think it is tab 9, right?
So it used to just give us the utter completion of the code
just based on the code base that we are currently using.
It could just charge the whole code base
and it would just utter complete whatever is written over there.
But it, in fact, creates code duplication, right?
Because what it is currently suggesting
is already implemented as another part of the code.
ken
So I get your point.
So basically, they just suggest it as what they already existing.
So if we want to improve that,
we have to explicitly tell them that,
please give me improved version
or we have to give them the improved version by ourselves.
Otherwise, they're consistent, which is good.
But they're just another copy and paste monsters.
Sudip Ghimire
Right.
So at this point, I just want to ask you some questions about general AI.
執筆と学びの意味
Sudip Ghimire
So do you have any thoughts about general AI?
I mean, currently we are just using specific AI, right?
For specific tasks.
And not just generic AI like C3PO, I previously said, right?
ken
Wow, that's a very great question.
And that's a broad topic.
And one thing I've recently thinking about general AI is,
you know, I write a lot.
I blog a lot.
I write so many private notes.
And I keep journaling for a few years as well.
And then one thing I found that I tinkering,
I tinkered by asking AI to input my blog post
and try to imitate my writing patterns and give me some drafts.
It does a great job.
Yes.
So let's say I ingest in my 50 blog posts about databases
and then ask Asian to come up with, you know,
new posts about, you know, Cloudflare, D1 and SQLite
based on, you know, not diving too much from my own taste,
which does a great job.
Yes.
And then at the same time point,
writing itself had a meaning to me.
After reading a lot,
after, you know, reading so many documentation,
soft score and through writing by my own hands,
it helped me understand the difficult concept better.
So writing itself has a great meaning for me
in term of learning something,
you know, understanding something.
So and I remember you told me like five minutes ago
that if we relied on AI too much,
we, our performance started degraded.
And I told you I agree with that
because if I ask Asian to start writing,
then I lose the steps in my learning process
to, you know, review my own thinking and self-review
and, you know, ponder what I did and learn from failure.
If I outsource that learning process to the AI
and then what am I doing?
So I realized that, you know,
I ask Asian to, you know, do a lot of productive stuff
like summarize this PDF
or just give me a better idea
or fix my type or whatever.
But I see the distinction between writing as a learning
and writing as a output.
Does it make sense?
Sudip Ghimire
Yeah, right.
Yeah, that makes sense.
I think we're in the same place
AIとデータ解析の探求
Sudip Ghimire
about the effect of AI, right?
ken
Yeah, I mean, as a human,
we need to keep evolving and keep learning
and learning itself is
I'm pretty sure you agree with me
because we have the same personality
but learning new thing is really fun, exciting
and energizing, right?
Inspiring.
And this is, we want to keep that value in my life.
So this is how I'm trying to struggling with,
you know, taking a distance between, you know, me and AI.
But, you know, I don't have any exact answer,
but that's what I'm thinking recently.
Sudip Ghimire
Yeah, yeah, that's right.
I'm studying.
So yeah, by the way, to be frank,
I'm also not a professional
Yearly Limb, you know, models engineer
Because, you know, until then, before early Limb was,
I mean, to be frank, if I remember correctly,
transformer models were first introduced in the year,
maybe around 2016.
I'm not sure.
Before that, all of our AI models
used to be something like CNNs, right?
Or especially neural networks, right?
Whether it is cascaded neural networks
or dense neural networks, whatever.
We used to have something like that.
And I was stuck into the LSTM.
It's like long short term memory.
So I used to do that for sequential data.
To be frank, when we were talking about the stock market,
I used to do it with the LSTM.
And now what I could see is like,
OK, instead of LSTM, if I just send all of those data,
I mean, feed all of those data
to current generative models, transformers,
I have started facing different issues.
I mean, something like, yeah,
instead of getting better result,
it started, I mean, giving its own feedback
because previously I said it
adds some layer of positional encoding, right?
So because of that positional encoding,
the same value of this stock market, for example,
has different bias at some point
and it creates completely different result at the end
and which really was not consistent with the current market.
So what I used to do was like,
I used to feed all the data from the year 2001 up to 2023
and check whether it is correct in the year 2024.
And LLMs, I think LSTMs were better than LLM until now.
Maybe they might add...
ken
Until now?
Sudip Ghimire
Yeah.
ken
So it's still beating LLM?
Sudip Ghimire
Yeah.
I mean, LLM might be correct at some specific domains,
but it's not yet ready for the general artificial intelligence.
That's where I found it, right?
ken
Can you say again, what does LSTM stand for?
Sudip Ghimire
Oh, yeah. LSTM means long short term memory.
So it's short term memory neural network.
So it's something like,
it takes a very recent data with higher bias
and less recent data with lower bias.
For example, stock market.
For example, I'm not sure what the current price is.
So let me just say some variable, right?
So placeholder company named X.
So the company might have,
for example, $100 at today's date, right?
So it was, for example, $50 one month before.
And if I see history data,
it might be $50 in the past year or maybe past decade, right?
So we cannot just take more bias of that $50 from the past decade,
but we should take more bias from the previous week or month.
So where LSTM really is useful.
So it's just a short term memory,
but it has effect from the longer term,
or let's say from the longer past,
but it has very few bias.
So eventually it will be almost equal to zero bias for very old data.
So that's where we need sequential database
and recent data for LSTM.
But, you know, in case of YALLM,
it just takes the data from the whole time frame,
for example, maybe from 2001.
And it will give you some prediction,
but it's still a matter, I think, in some domains.
So it is obviously growing,
but I think it takes a lot of time.
ken
Yeah, definitely.
Yeah, this is a great call out.
LLM is in the trend,
but it's not silver bullet, right?
It's a classic word to say that,
but it's not for everything.
And we have to also think about the cost performance as well.
Let's say LLM and LSTM can do the same performance,
but how much cost do I put onto this LLM prompt or whatever, right?
So it depends on like in practical term,
if we started building everything on LLM,
who pays for that, right?
AIと経済的影響の議論
Sudip Ghimire
Yeah, that's really a good question.
Because, you know, I still remember,
I mean, feeding those inputs to a single LLM model,
I mean, it has not only hundreds,
but thousands of neural networks to pass, right?
But if we are just solving a basic problem,
for example, on a specific domain,
not a multi-model, right?
So I think LLM is just a waste of maybe time,
resources, everything, right?
Because we're just trying to,
for example, if we want to maybe bring a bottle of water,
then we do not need the whole cargo truck, right?
We can just do it with a bicycle, right?
ken
I love the analogy, yeah.
Sudip Ghimire
So that's where current LLM is going,
maybe out of track for most of people,
because I could see most of people are more fascinated
by the results of the LLMs,
but still maybe all of them might not be suitable
for those things, right?
So basically we're wasting a lot of money and resources
because the more resources we waste on the technology,
the more money and eventually it will,
maybe, I'm not sure,
but it will collapse our economy
because everything will...
I mean, if we solve everything with the help of AI,
then I think all of our financial resources
should be for AI rather than the human development
or maybe for infrastructural development, right?
ken
That's not ideal.
Sudip Ghimire
Maybe that might not happen in the near future,
but who knows, right?
エピソードの締めくくり
ken
Yeah, who knows?
Yeah, cool.
I feel like we can keep talking for another one more few hours.
We just covered both positive side and negative side,
so I feel like it's a great point to wrap up about AI as well.
And is there anything that you want to shout out in the last word?
Are we closing?
Sounds good.
Sudip Ghimire
Yeah, well, I think for me,
I think I have said what I needed to say.
So if you have any confusions
or maybe if I might be the one who could help you with some confusions,
otherwise, I think, yeah, that's all from my side.
ken
Yeah, if you have any follow-up question, talk to Sareb.
Sudip Ghimire
Yeah.
ken
You know, this is not our last recording,
so let's carry on
and please bring in the same topic or different topic.
Let's keep it going.
So yeah, but for today's episode, let's get close in.
So yeah, I really appreciate you, you know,
being with us on Mono Tech Talk again.
And thank you for coming today
and sharing valuable insight on AI
and asking so many good questions
when I was talking about cloud-heavy ones.
So as usual, I really enjoyed talking with you.
Let's do another recording as well.
Sudip Ghimire
Right.
So it really was an interesting Tech Talk for me.
So yeah, see you again sometime.
ken
I should change the podcast name again.
Right now, Kaz and me are hosting from London,
but now why we are inviting guests from Nepal?
Sudip Ghimire
Right.
ken
It's not a good name, I know.
Sudip Ghimire
Okay.
ken
So yeah, and also for our listeners,
thank you everyone for listening
and we will see you next time.
So take care, everyone.
Bye-bye.
