This would be an interesting approach if the course supplied a custom Harness (perhaps in place of a textbook) and this was part of the instruction set inside of it. As a standalone thing you ask students to import into their agent, seems unlikely to work.
To be fair, shipping these guidelines as AGENTS.md/CLAUDE.md in the repo that contains the assignments will make it so that agents will pick this up without needing students to opt in explicitly. Seems like a reasonable first step to me
This seems somewhat sensible to me - the genie _is_ out of the bottle, and students absolutely will use AI agents to finish assignments without learning a thing, but there is some value to showing how agents can be used as teaching tools and what healthy use _can_ look like
Same issue as with cliffnotes. Easy way out means the easy way will be taken. Unless, you actually design a decent assignment or exam. In person essays or exams, heavily weighted, you are simply screwed if you didn't study the old fashioned way. A couple of my more serious classes were like this: no homework, no projects, entire grade based on 3 exams. That put the fear of whatever diety you subscribe to into you like nothing else to study hard and not fall behind. One bad exam you can't really come back from. Better luck next year when you retake it. Or, you dig in like hell.
The insidious thing here is that students can think they're studying and practicing by chatting with an AI "tutor", which shifts them into a passive observation role that's no better than watching YouTube videos.
It turns out that it's much less memorable if you're too "clear and helpful", so nothing helpful sticks for students. A good teacher (tutor, educator, pick a word) challenges students and makes them the right amount of uncomfortable.
Agreed. I don't know how they plan to enforce this but this is way better than some other articles that have come up indicating educational bans on AI use, in-person proctoring, verbal assessments, pen and paper exams etc. This is the first attempt at an approach I've seen that doesn't seek to isolate education from reality; students that are effective at integrating AI into their work and actually understand what they're doing are going to get jobs, which is ultimately the goal of school.
They're only cheating themselves in a world that increasingly cares about knowledge (market trend of seniors being preferable hires to fresh out of school juniors) and not the piece of paper that "proved" you had such knowledge.
Hah, I like that these are presented as a CLAUDE.md.
(They have the same content duplicated in an AGENTS.md as well - I really wish Anthropic would hurry up and teach Claude Code to check for that file too.)
This is a very good baseline for future courses to build on, there would always be a group that wants to jailbreak this and thats okay, but have baseline agent support learning is needed in this ai first world.
This is interesting. I don't know how the AI agent guidelines will be enforced because there will always be a model outside the curriculum that a student can use to bypass the guidelines. Encouraging academic integrity is useful but requires the student to buy into the idea that they are paying for an education, not a diploma. This is a tough problem and I have been wondering how CS departments are incorporating AI into the curriculum while encouraging appropriate use in a learning environment.
Stanford has an honour code. Meant no oversight even during exams. Worked surprisingly well when I was there. The flipside is, if you’re ever caught cheating, there are no second chances.
I imagine this applies here, too, if they want to enforce it strictly.
How could you tell? I proctored. People cheat pretty frequently and other students are none the wiser. It really takes like 4 proctors if you want to do it right. Even then I'm sure the clever ones are slipping through. These were scantron though. Short response/essay format you'd be screwed if you didn't know your stuff.
You mean it worked well for cheaters right? The more I learn about these "honor codes" the more I realize how sheltered these American elites have become.
In an ideal world guidelines should be suggestions for those willing to make the best of the course and improve as a person and professional. However a degree has real world value and repercussions, so enabling someone incompetent to do a dangerous job can put innocent lives in jeopardy. It's tough, but I hope in time we learn how to live with this new tech.
yeah I don't think that's going to work - it would be kind of like "we're releasing model answers to all assignments but please only use them as a teaching aid and don't copy from them"
best to
a) adapt assignments so that agents are bad at producing solutions
b) have more scenarios where students have to do things in controlled environments. Universities managed to adapt to 'any solution you need is readily available online' so I don't think it will be that different to have several times a month/year where students have to go into a room with nothing but pencil and paper to prove what knowledge they have vs what they have the skills to access
I did most of my CS class tests this way within the last year. It’s not that bad because prof doesn’t care about syntax so much (unless that’s what we’re testing on of course) and details, but wanting instead to make sure we understand broader concepts.
I agree it's not a complete solution. But as those don't exist as a society we are looking for a step function in the right direction. and IMO this is one such step. You may disagree that it's not a very large step, but I would argue it's still in the right direction therefore it is neccesary, especially in education space, and I'm happy to see someone publishing at attempt.
Is this all an elite educational institution with about $50bil in assets could muster, lol? This is completely and utterly unenforceable, and such, worthless.
There really needs to be diversity in delivery styles for different modules of courses according to their aims, with 'ai access' as a key variable.
If AI is allowed, it should be based on $x of usage/student, with an audit trail to prove no external funding was used, and module aims based on using AI to the max while conserving token use. Like actually creating wild, ambitious shit which takes cutting edge services to the max.
If AI is not allowed for a module, then it really needs to go back to the old skool, with handwritten exams, or coding using old machines and textbooks. Some skills, techniques, etc, really do need drilling.
Straddling the middle will help nobody, result in accusations, increase the burden on teaching staff, and result in a course without a realistic focus.
Though I guess if you're a big brand university, you don't really need to care about innovating. The money will keep pouring in. The whole further education sector is in dire need of a shake up.
I don’t really know why this is getting downvoted. It’s clear that higher education is degrading because of easy to reach AI solutions that have no type of penalties for use.
During my undergrad it was normal to see people refer to Chegg solutions to get their answers, or as a friend for theirs.
Maybe there’s a reason my first CS professor wrote out Java code with pencil and paper I guess.
I really like this. I'm currently doing a part time BSc and my current module explicitly allows AI usage as long as you 'cite it'. The guidelines are out of date in that they assume you are using a chatbot and not a coding harness. The temptation to have claude write all my pandas code has become too difficult for my self control, but at the same time I actively feel my education is suffering from using it. As I write my final paper I am thankful that I at least despise AI writing too much to use it for the actual marked assessment, but I still feel that I have cheated myself out of part of my education and probably wasted a lot of time going fast in the wrong direction because generating data frames, graphs, statistics, etc. is just so easy with claude
Students are struggling to get work after graduating because they're dropped into a competitive environment. Ideals aren't enough to get jobs in the current environment.
Universities should be places which are at the bleeding edge of development and providing society with the best new ideas/tech, etc has to offer. Junior workers should be hotbeds of exciting talent which have the ability to revolutionise industries.
By creating such milquetoast environments to study in, which are seemingly scared or unable to prepare people for the future, students are being done a disservice.
Far too many people are far too comfortable with their cushty positions, and it's not doing the youth any favours.
This is ridiculous. The genie is not going to go back into the bottle. This is the equivalent of "you wouldn't download a car". (Yes, we would.)
The solution is to scale the difficulty of the objective measures. Expect far more from students.
Reorient the university around physical laboratories and timesharing resources no single student could afford. It's already like this in many STEM disciplines.
More internships, more networking, more large projects. Less trivial tests of knowledge and credentialism.
Seeing my own kids (teens) go through some of this, I'm becoming slightly less pessimistic as it all shakes out. Among their peer groups there does seem to be an opinion forming that sure, anyone can just ask ChatGPT for quick answers on assignments, but actually knowing stuff is a bit of a "flex" that's respected.
300 years ago when I was in high school I had a friend choose to go the HVAC trade school route instead of college. He chose the hardest school in the country where they did most things manually so that students understood how things work. It removed the "magic" some tools provide. I was pretty impressed he was wise enough to do that. He's exceptional at his job by the way.
I think we have a tendency to think the worst of your people. They frequently surprise me though.
When calorie dense food and gas powered vehicles came on the scene, humans (generally) got fat and out of shape. "Why eat that salad and go for a run?" one might say, "This cheesecake tastes much better and I can just drive wherever I want to go."
Getting fat is one thing, but getting stupid is another, and I really fear for the future of humanity when it becomes so easy to sidestep the processes that let us actually learn and grow because stuff like "using agent ai coding is trivial".
using a coding tool is trivial, correct. so is using a microwave oven. you need a certain level of person to know if what came out of it was Michelin-star or not and I do not think Stanford is going for Hot Pockets here.
Seems like a pretty close copy of Carson's (of HTMX fame) agent.md from 5 months ago
https://gist.github.com/1cg/a6c6f2276a1fe5ee172282580a44a7ac
They reference the gist of 1cg in the honor code section of CS336.
https://cs336.stanford.edu/
This would be an interesting approach if the course supplied a custom Harness (perhaps in place of a textbook) and this was part of the instruction set inside of it. As a standalone thing you ask students to import into their agent, seems unlikely to work.
To be fair, shipping these guidelines as AGENTS.md/CLAUDE.md in the repo that contains the assignments will make it so that agents will pick this up without needing students to opt in explicitly. Seems like a reasonable first step to me
This seems somewhat sensible to me - the genie _is_ out of the bottle, and students absolutely will use AI agents to finish assignments without learning a thing, but there is some value to showing how agents can be used as teaching tools and what healthy use _can_ look like
Same issue as with cliffnotes. Easy way out means the easy way will be taken. Unless, you actually design a decent assignment or exam. In person essays or exams, heavily weighted, you are simply screwed if you didn't study the old fashioned way. A couple of my more serious classes were like this: no homework, no projects, entire grade based on 3 exams. That put the fear of whatever diety you subscribe to into you like nothing else to study hard and not fall behind. One bad exam you can't really come back from. Better luck next year when you retake it. Or, you dig in like hell.
The insidious thing here is that students can think they're studying and practicing by chatting with an AI "tutor", which shifts them into a passive observation role that's no better than watching YouTube videos.
It turns out that it's much less memorable if you're too "clear and helpful", so nothing helpful sticks for students. A good teacher (tutor, educator, pick a word) challenges students and makes them the right amount of uncomfortable.
Agreed. I don't know how they plan to enforce this but this is way better than some other articles that have come up indicating educational bans on AI use, in-person proctoring, verbal assessments, pen and paper exams etc. This is the first attempt at an approach I've seen that doesn't seek to isolate education from reality; students that are effective at integrating AI into their work and actually understand what they're doing are going to get jobs, which is ultimately the goal of school.
They're only cheating themselves in a world that increasingly cares about knowledge (market trend of seniors being preferable hires to fresh out of school juniors) and not the piece of paper that "proved" you had such knowledge.
Hah, I like that these are presented as a CLAUDE.md.
(They have the same content duplicated in an AGENTS.md as well - I really wish Anthropic would hurry up and teach Claude Code to check for that file too.)
We symlink AGENTS.md and CLAUDE.md to a single file in our repo
You can also include other md files like AGENTS.md in CLAUDE.md:
They won't, because forcing the file to be named after their product is an intentional marketing choice. Free advertising on every repo that has it.
I wouldn't hold my breath.
This is such a realistic balance between completely banning coding agents and embracing the spirit of higher education
This is a very good baseline for future courses to build on, there would always be a group that wants to jailbreak this and thats okay, but have baseline agent support learning is needed in this ai first world.
Jailbreaking isn’t even needed - you can just modify the file
I just took a C1 Spanish class and it had almost exactly the same instructions. Hmmm and I do not wonder why...
This is interesting. I don't know how the AI agent guidelines will be enforced because there will always be a model outside the curriculum that a student can use to bypass the guidelines. Encouraging academic integrity is useful but requires the student to buy into the idea that they are paying for an education, not a diploma. This is a tough problem and I have been wondering how CS departments are incorporating AI into the curriculum while encouraging appropriate use in a learning environment.
Stanford has an honour code. Meant no oversight even during exams. Worked surprisingly well when I was there. The flipside is, if you’re ever caught cheating, there are no second chances.
I imagine this applies here, too, if they want to enforce it strictly.
>Worked surprisingly well when I was there.
How could you tell? I proctored. People cheat pretty frequently and other students are none the wiser. It really takes like 4 proctors if you want to do it right. Even then I'm sure the clever ones are slipping through. These were scantron though. Short response/essay format you'd be screwed if you didn't know your stuff.
Marc Tessier-Lavigne was Stanford's president from 2016 to 2023. Not sure if the honor code means anything nowadays.
You mean it worked well for cheaters right? The more I learn about these "honor codes" the more I realize how sheltered these American elites have become.
Well, no amount of instructions would work if the student has no intention to learn anything.
In an ideal world guidelines should be suggestions for those willing to make the best of the course and improve as a person and professional. However a degree has real world value and repercussions, so enabling someone incompetent to do a dangerous job can put innocent lives in jeopardy. It's tough, but I hope in time we learn how to live with this new tech.
Interesting. It makes me think of the idea of fighting piracy by providing a solid legal alternative through streaming platforms, etc.
I think these are based on the one I posted a while back:
https://gist.github.com/1cg/a6c6f2276a1fe5ee172282580a44a7ac
Yes absolutely! We linked your version inside the extended AI policy document, but forgot to add it to our website cs336.stanford.edu
I'm definitely going to use a variation of it for learning new programming languages.
yeah I don't think that's going to work - it would be kind of like "we're releasing model answers to all assignments but please only use them as a teaching aid and don't copy from them"
best to
a) adapt assignments so that agents are bad at producing solutions
b) have more scenarios where students have to do things in controlled environments. Universities managed to adapt to 'any solution you need is readily available online' so I don't think it will be that different to have several times a month/year where students have to go into a room with nothing but pencil and paper to prove what knowledge they have vs what they have the skills to access
Laptop without internet access, sure. Pencil and paper? that is brutal :)
I did most of my CS class tests this way within the last year. It’s not that bad because prof doesn’t care about syntax so much (unless that’s what we’re testing on of course) and details, but wanting instead to make sure we understand broader concepts.
I agree it's not a complete solution. But as those don't exist as a society we are looking for a step function in the right direction. and IMO this is one such step. You may disagree that it's not a very large step, but I would argue it's still in the right direction therefore it is neccesary, especially in education space, and I'm happy to see someone publishing at attempt.
> What AI Agents SHOULD NOT Do
> * Run bash commands
Students who prefer to use zsh keep winning.
zsh is fine, but I prefer fish. It has a funnier name!
Is this all an elite educational institution with about $50bil in assets could muster, lol? This is completely and utterly unenforceable, and such, worthless.
There really needs to be diversity in delivery styles for different modules of courses according to their aims, with 'ai access' as a key variable.
If AI is allowed, it should be based on $x of usage/student, with an audit trail to prove no external funding was used, and module aims based on using AI to the max while conserving token use. Like actually creating wild, ambitious shit which takes cutting edge services to the max.
If AI is not allowed for a module, then it really needs to go back to the old skool, with handwritten exams, or coding using old machines and textbooks. Some skills, techniques, etc, really do need drilling.
Straddling the middle will help nobody, result in accusations, increase the burden on teaching staff, and result in a course without a realistic focus.
Though I guess if you're a big brand university, you don't really need to care about innovating. The money will keep pouring in. The whole further education sector is in dire need of a shake up.
I don’t really know why this is getting downvoted. It’s clear that higher education is degrading because of easy to reach AI solutions that have no type of penalties for use.
During my undergrad it was normal to see people refer to Chegg solutions to get their answers, or as a friend for theirs.
Maybe there’s a reason my first CS professor wrote out Java code with pencil and paper I guess.
I really like this. I'm currently doing a part time BSc and my current module explicitly allows AI usage as long as you 'cite it'. The guidelines are out of date in that they assume you are using a chatbot and not a coding harness. The temptation to have claude write all my pandas code has become too difficult for my self control, but at the same time I actively feel my education is suffering from using it. As I write my final paper I am thankful that I at least despise AI writing too much to use it for the actual marked assessment, but I still feel that I have cheated myself out of part of my education and probably wasted a lot of time going fast in the wrong direction because generating data frames, graphs, statistics, etc. is just so easy with claude
I am really baffled by the comments in the spirit of "this is unenforceable, and therefore worthless".
I bet most people would not steal even if they knew they could get away with it.
Students are struggling to get work after graduating because they're dropped into a competitive environment. Ideals aren't enough to get jobs in the current environment.
Universities should be places which are at the bleeding edge of development and providing society with the best new ideas/tech, etc has to offer. Junior workers should be hotbeds of exciting talent which have the ability to revolutionise industries.
By creating such milquetoast environments to study in, which are seemingly scared or unable to prepare people for the future, students are being done a disservice.
Far too many people are far too comfortable with their cushty positions, and it's not doing the youth any favours.
I mean, some would say that's how this whole thing got started.
And, yes students are going to follow it....
Related:
CS336: Language Modeling from Scratch
https://news.ycombinator.com/item?id=48357075
This is ridiculous. The genie is not going to go back into the bottle. This is the equivalent of "you wouldn't download a car". (Yes, we would.)
The solution is to scale the difficulty of the objective measures. Expect far more from students.
Reorient the university around physical laboratories and timesharing resources no single student could afford. It's already like this in many STEM disciplines.
More internships, more networking, more large projects. Less trivial tests of knowledge and credentialism.
good intention but useless let's be real
Seeing my own kids (teens) go through some of this, I'm becoming slightly less pessimistic as it all shakes out. Among their peer groups there does seem to be an opinion forming that sure, anyone can just ask ChatGPT for quick answers on assignments, but actually knowing stuff is a bit of a "flex" that's respected.
300 years ago when I was in high school I had a friend choose to go the HVAC trade school route instead of college. He chose the hardest school in the country where they did most things manually so that students understood how things work. It removed the "magic" some tools provide. I was pretty impressed he was wise enough to do that. He's exceptional at his job by the way.
I think we have a tendency to think the worst of your people. They frequently surprise me though.
Teens also fucking hate AI, on a cultural-ideological level.
I think they may hate what it may be doing to their future outlooks but they use it as much as they do social media
Yeah, that's exactly why I added the second clause.
Nevertheless. The peer pressure is to be anti-AI.
I always wonder why there is such course. Using agent ai coding tool is trivial.
When calorie dense food and gas powered vehicles came on the scene, humans (generally) got fat and out of shape. "Why eat that salad and go for a run?" one might say, "This cheesecake tastes much better and I can just drive wherever I want to go."
Getting fat is one thing, but getting stupid is another, and I really fear for the future of humanity when it becomes so easy to sidestep the processes that let us actually learn and grow because stuff like "using agent ai coding is trivial".
There's different skills at play, and they're both as valuable as each other.
They shouldn't be thrown into a big soup with shaky aims.
We still - as a society - manage to have PE and driving as different subjects. The same can equally apply here.
using a coding tool is trivial, correct. so is using a microwave oven. you need a certain level of person to know if what came out of it was Michelin-star or not and I do not think Stanford is going for Hot Pockets here.
Pangram reports as 100% AI generated. Makes sense for a README, but a tad bit funny given that their students must hand-write code