Getting AI agents as team members

Posted by JuanistaD@reddit | ExperiencedDevs | View on Reddit | 15 comments

I work at a bank and we’ve been quite slow with the adaption of AI. Last week, senior management announced plans to have AI agents in every development team within two years, where they will count as additional team members. Basically they are replacing our external colleagues (self employed and consultants, etc).

The idea is that the agents will review our PR’s and some extra small stuff.

Anyone had some experiences that you could share about agent team members? The idea is still quite new to me. I wonder what the future will bring :).

Edit: we work with GitHub copilot based on ChatGPT. The agents will also run on that.

[-]

mrothro@reddit

One easy and low-risk thing I do is have agents do PRs. For anything critical, they should not be the *only* reviewer, but they are very good at finding easy things that would just otherwise be a drag on a human reviewer. Think of them like enhanced lint.

Three things that made this really effective for me: they should 1) be agentic with the ability to examine other parts of the code to ensure consistency; 2) be prompted to verify consistency with the project conventions; and 3) have access to the plan and design documents that specify the PR requirements so they can verify compliance.

My feature requests come with deterministic tests. My agents have the tools to run those and are prompted to do it so I know the code works and meets the AC.

This approach ensures the code meets both the AC and has no obvious defects before it gets to a human reviewer. Also, it lets the developers get pretty immediate feedback on issues without having to wait for a senior to do a review.

[-]

sole-it@reddit

Ha, regular bank employee could go to jail if they messed up, who is going to take the liability if the agents make a mistake? Definitely not that senior management person.

[-]

ObeseBumblebee@reddit

We use GitHub copilot agents and have found it successful.

My work process is this:

Assign my name to a story.

Assign copilot.

Give it some initial instructions within GitHub UI upon assigning it. Usually I'll tell it the exact location where to start looking to make it's changes. This helps it figure things out faster.

Then once it's assigned GitHub will display that a copilot session is active.

You can view this session and baby sit the AI working in real time and watch it think and code out loud.

You can also prompt it in real time to steer it if it goes down the wrong path.

Once it's done it will create a pr. Which you can review and directly tag it with "@copilot" to start a new session to address your feedback. You can also watch and steer this session.

The important thing to remember is you are the driver still. You're assigned to the story and you're responsible for the code that makes it through PR.

AI is still just a tool and it's not as smart as you. Just faster.

[-]

Leopatto@reddit

Look you're in a sub where there's "supposedly" experienced developers, having a bot that's capable of doing your job means that your ass is on the line and people don't like that as that’s their livelihood.

[-]

ObeseBumblebee@reddit

I'm not worried about it. AI agents can't do the job on their own and frequently require an expert like me steering them to prevent disaster.

AI is simply not capable of keeping all the domain knowledge and business logic i have in my head. Not affordably anyway. Too much tokens to do all that

[-]

TalesfromCryptKeeper@reddit

You gotta realize that the junior-senior pipeline to get to your level of expertise is irrevocably damaged by this. It feels like so many people on this sub and elsewhere are living in the present only and don't foresee long term consequences, sadly.

[-]

ObeseBumblebee@reddit

I don't control any of that unfortunately.

I'm just using the tools bossman tells me to use.

[-]

TalesfromCryptKeeper@reddit

I know.

[-]

kruvii@reddit

Our IDP, Port (though I'm sure you can do this with Backstage), basically sets up agent as developers. Easy way to understand what agents are doing and touching. Also good for putting guardrails on what they can't do.

[-]

GumboSamson@reddit

Having agents-as-code-reviewers has really helped my teams.

We still have human reviewers for every PR too—the agents just do the first few passes and it means that by the time a merge request is getting reviewed by a human, all of the low-hanging fruit has been dealt with. This means that people get fast feedback on their PRs but also people don’t need to spend as much time on PRs. Win/win.

Getting the agents to do code reviews well didn’t come for free. We had to write the AGENTS.md files, the copilot-instructions.md files, various skills. But this forced us to have conversations about what our coding standards actually are and that was a valuable exercise, too.

[-]