Context, verified sources, and human review remain essential as teams build AI tools.
Sponsored by Radical Pricing by the Radical CPA, Jody Padar | See Today’s Special Offer
Subscribe to CPA Trendlines podcasts anywhere: Apple, Google/YouTube, Spotify, iHeart, Deezer, Amazon Music, Audible, Player FM, Audacy, RSS.
Innovation Insights
With Donny Shimamoto and Liz Mason
Center for Accounting Transformation
A tax firm can build an AI agent to organize documents, flag missing information, and track unfinished returns. The harder assignment is deciding what that agent should never do on its own. That question drives this episode of Innovation Insights, a Center for Accounting Transformation podcast conversation between hosts Donny Shimamoto, CPA.CITP, CGMA, and Liz Mason, CPA.
As smaller firms experiment with building their own tax tools, the two examine what separates a useful automation from a costly lapse in security, accuracy, or professional responsibility.
Shimamoto is founder and managing director of IntrapriseTechKnowlogies LLC and founder and inspiration architect for the Center for Accounting Transformation. Mason is co-founder and CEO of TaxFigure, founder and CEO of High Rock Accounting, a member of Intuit’s Partner Council, a board member of AuditMiner, and CEO and founder of Rebel Rock.
Their message gives firms room to experiment, but it also gives that experimentation a clear starting point: Understand the data, the workflow, and the professional obligations before building.
Security belongs at the beginning
For Mason, the first design decision concerns client information. She describes TaxFigure’s approach as using a privacy vault to remove personally identifiable information before data reaches a large language model. The broader lesson extends beyond any single product. Firms need to understand where client data travels, who can access it, how it is retained, and whether the applicable vendor terms and technical controls support the intended use. A paid subscription alone does not answer those questions.
Shimamoto points out that the exposure can extend beyond tax returns to documents used in client accounting services, including W-9 forms and bank account information. The hosts also discuss less obvious routes for sensitive information to leave a controlled environment, such as screenshots, shared conversations, and monitoring tools. Mason urges firms to consider Section 7216, Circular 230, and their own policies before implementing AI. The distinction matters: Section 7216 restricts unauthorized use or disclosure of tax return information; firms must determine whether a particular use is permitted or requires taxpayer consent. It is not simply an offshore outsourcing issue.
The IRS Office of Professional Responsibility’s introductory AI guidelines reinforce the need for secure data handling, documented verification, and professional oversight. AI does not remove a practitioner’s responsibility to validate the work.
The biggest model is not always the best fit
Security is only part of the architecture. Mason also challenges the assumption that every task needs the most powerful available AI model. She describes a “council of models” that routes straightforward lookups to a smaller, self-hosted model and sends more complex questions to larger models. A separate review process checks citations and compares responses before results return to the user.
The approach aims to match the resources to the assignment, controlling cost and compute use while supporting accuracy. Shimamoto compares the decision to choosing equipment: A job that calls for a shovel does not necessarily require a backhoe.
For firms building their own tools, that means defining the task before selecting the technology. A narrowly scoped workflow supported by authoritative information may be more useful than an expensive model asked to solve an ill-defined problem. Model-to-model checking adds a layer of review, but it does not eliminate the need for a professional to verify the result.
Context still comes from the professional
Even a well-designed system can produce an unsuitable answer when it lacks the facts. Mason compares the challenge to working with clients who do not know which information matters to a tax decision. A client may omit a property sale, another business interest, or a document because its relevance is not obvious. An AI system cannot reliably fill those gaps simply because its answer sounds complete. The accountant supplies the context, asks follow-up questions, and determines whether a recommendation fits the client’s circumstances.
“You’re the professional. You have the expertise,” Mason says.
The hosts emphasize that client relationships are part of the technical solution. Conversations reveal information that a form or an automated intake process may miss. That understanding helps a practitioner distinguish a defensible position from advice that overlooks a material fact.
Shimamoto frames the final review around a direct question: “Does this make sense for this client?”
Administrative work offers a practical starting point
Mason identifies several uses firms can explore: sorting incoming documents into client folders, identifying open requests, monitoring project status, and organizing completed returns. Dashboards can also flag work that appears stalled despite having the necessary documents. These applications can reduce the effort spent finding files and tracking progress. They still require security controls, appropriate access, and a process for checking results. Mason advises particular caution before giving an agent direct access to tax-preparation software. The distinction is between assisting the workflow and handing over a consequential decision. Administrative automation calls for a different kind of review than tax advice, but it still handles information the firm is responsible for protecting.
The payoff is more room for advisory work
Mason sees the greatest opportunity in using AI to prepare research and organize ideas so professionals can spend more time evaluating them and talking with clients.
“I don’t think there’s an ethics issue to using AI to elevate what we do,” she says. “I think there’s an ethics issue to using AI to replace what we do.”
Shimamoto connects those efficiency gains to the business model. When technology reduces the time required for valuable advisory work, he says, fixed or value-based pricing becomes increasingly relevant.
For firms considering their first build, the episode points toward a manageable assignment: Choose a specific task, establish the safeguards, and decide where human review belongs. The opportunity is to spend less time moving information and more time applying the expertise clients need.
10 Key Takeaways
- Start with safeguards. Map the workflow, data access, security controls, and review responsibilities before building an AI tool.
- Evaluate consent requirements. Determine how Section 7216 applies to the intended use or disclosure, including whether an exception or taxpayer consent is required.
- Check the actual agreement. Review vendor terms, retention practices, and configurations for the specific service and intended data use.
- Match the model to the task. Straightforward lookups may call for smaller models; complex questions may justify more capable systems.
- Use authoritative information. Define the sources an AI tool can consult, and verify citations and interpretations.
- Supply the context. Professional expertise helps identify the facts an AI system needs and the information a client may omit.
- Begin with a defined administrative task. Document sorting, missing-item tracking, status monitoring, and return organization offer practical starting points.
- Preserve professional judgment. Review AI-assisted work for accuracy, client fit, and defensibility before relying on it.
- Protect the client relationship. Use time savings to ask better questions and understand the client’s circumstances.
- Revisit pricing as work changes. Consider how fixed or value-based pricing reflects the value of advisory work when preparation time falls.
Transcript
(Transcripts are not edited for spelling, grammar, or punctuation.)
Donny Shimamoto 00:42
Alright, welcome everyone back to another episode of Innovation Insights, where we are really seeking to demystify these innovations that are coming out. And today, I’m really glad to have Liz with me here, and we’re going to talk about demystifying using AI to create tax apps or using it in the tax space, and so Liz, you want to give us a little background on on what you do in this area?
Liz Mason 01:07
Yeah, absolutely. So I have been a CPA for 20 years, which is terrifying and exciting all at the same time. And I started my career in tax specifically, and I actually was part of the team that helped build the paperless system for tax for Grant Thornton and implement that. So my history in tax technology goes back to paper-based file folders, which is also kind of terrifying. And now we’re here in the age of AI, and I’ve been through the years working on tax technology, streamlining that. How do we use it? How do we build our own integrations and innovations, and use workflow automation and what all the IRS regulations are around that? And of course, like the code of ethical conduct, how do we integrate that? All the way to last year, where I launched a very specific tax technology tool that is AI called Tax Figure, and so now I have been building throughout the years, and very specifically building in the age of AI in the tax industry.
Donny Shimamoto 02:12
Well, well, that’s where I I’m really thankful that you’re you’re willing to share. You know, don’t share your trade secrets, of course, but share some of the lessons learned in creating that tech that technology, because from what I saw, that is some awesome stuff that’s going on there.
Liz Mason 02:28
It is fascinating, and anyone that tells you that there are you know trade secrets in this day and age is lying, because the way that you can build and the quickness in which you’re able to do that, it’s really about how do you build the right the right stack of technology and the right feature sets that protect the industry and do the right thing versus like let’s build a big technology boat, which is not a thing that really exists in the same way it used to exist.
Donny Shimamoto 02:57
I love that. I love that. So I building off of that, I do want to narrow the scope of our conversation a bit because what I’m seeing is I’m seeing a lot of smaller firms, and I’m seeing people encourage some smaller firms. Hey, you can just grab these vibe coding tools. You can create your scripts. You can create your own agents, and you know you don’t have to pay any vendor for it. It’s kind of part of the AI
Liz Mason 03:26
vendor, which sometimes is actually more expensive because you’re using frontier models, which are wildly inefficient for what you probably need to do. But we can get into that later. Sorry, continue. I love already.
Donny Shimamoto 03:35
You’re already coming out with the great advice. So yeah, so that so that’s kind of where I want to I want to constrain us a little bit. That that if if if the listener out there, if you’re thinking about it, if you’ve got someone at your firm that you’re like this person’s a great whiz on doing this stuff, and you don’t have to be an IT person because they can do the vibe coding whatever they want. That that’s kind of where I want to take this conversation. Is what should these firms? What are some lessons learned from what you guys have been doing that you can share with them to make sure they get on the right path? And then, what are some of the red flags where if if they start to see these things or guardrails, what what should they be aware of? And I love that you’ve already brought the ethics because you know I love to talk about that side too. So
Liz Mason 04:14
yeah, so first intentionally we’re demystifying this, right? So let’s start at the basics. For me, I always start at vocabulary definitions because if we’re not speaking the same language, then we can’t understand each other, right? So down to basics. When I say something like frontier models, I mean the most up to date, biggest models that the you know OpenAI and Anthropic are producing for you, or even Gemini, right? Like we’re looking at the biggest, most cost-consuming models. An LLM is a large language model. All of these AI models are large language models. They interact in language. Some other definitions, I think, agentic AI and agent is very specifically an AI that you have. Programmed and trained to do specific tasks inside of a workflow, and it is supposed to be contained in a in a very specific vertical. Then you have harnesses, which is how you build the structure around utilizing these language models so that it does what you want it to do versus like goes and and runs the gambit of the universe, which it could potentially do. All right, I think that’s like probably a good basic definition start for me to answer the question. And so, you know, we have learned a lot of lessons. And one of the things that I think is highly important when you’re thinking about tax technology and utilizing AI is complying with IRS Regulation 7216, and we think about this from the perspective of foreign individuals or hiring contractors or where information is going. If it’s on your servers or if it’s with a technology provider, and historically speaking, our SaaS tools, as long as they had passed SOC two and they had the right level of encryption, did not require a 7216 disclosure, and that’s because they’re deterministic code. Another definition, okay? Deterministic code. It is if this, then that. Right? We’ve all done this in Excel. We’ve put together if things. It it has one answer. It is black and white. It does not make decisions on your behalf. There is no judgment happening there. There is no potential for information to go elsewhere because it is contained very specifically inside of a vaulted and encrypted ecosystem. Traditional SaaS tools totally fine. AI tools don’t really function in the same confines, right? And so when you start thinking about like requirements for this, the IRS put out a great interpretation of this, saying if you’re using AI with your client’s data, you need to get a 7216 disclosure. You also need to have a process very specific to understanding AI that includes review by a human that is certified on every single piece of information coming out of that, and then you need to like think holistically around the code of ethics if you’re a CPA firm. And I would argue that anybody in this profession should be utilizing that code of ethics, regardless of if they’re a registered CPA firm or not. I see you nodding. I’m glad you agree with that.
Donny Shimamoto 07:17
Definitely.
Liz Mason 07:18
And it’s you know the framework of understanding how do we do this safely. So foundationally, I think safety is number one, and so that’s the first thing we built at Taxfigure was a proprietary OCR that reads tax documents at a 99.9% accuracy, which is actually way harder than you would think it is. And LLMs don’t do the best job at reading tax documents specifically because of the way they’re structured, because they’re not reading it visually; they’re reading the actual words on it and translating that into a language called Markdown, which is effectively just text that they use to interact with the world. Another definition. So, anyways, Paul, you you said first thing to think about safety, security, 7216, making sure that you are complying with all of all of those regulations and the code of ethics.
Donny Shimamoto 08:05
Ooh, I love that you started with that because normally people think of it as well. Let me run my pilot. Let me do my things. Okay, then let me figure out whether any guardrails actually apply to this. And you are actually saying, wait, start with the guardrails first.
Liz Mason 08:19
Yes,
Donny Shimamoto 08:19
and make sure you understand what all these requirements are. And I love that you brought 7216 into this because most people think, well, I only need that if I’m going to throw stuff offshore. But it it really applies anytime I’m using the the tax information to not directly for the tax return itself,
Liz Mason 08:37
right? And using it through AI accounts that way, like that’s the way the IRS is interpreting it. And quite frankly, I think that’s the right answer because AI. While you know you think, oh, I’m just putting it into another program, and it feels very similar in that regard, it is not similar in that regard. And there are layers of judgment and decision making that large language models make on your behalf that you may not be privy to, and the way they utilize the data. Well, even if you have you know the highest level enterprise account on Claude, and I’ve had this argument with many accountants that I don’t think quite understand the terms and conditions of it. You’re not supposed to put tax data into there first off, like that violates their T’s and C’s. Any private PII, so personally identifiable information, when it comes to tax returns or other regulated industries, are not supposed to go onto the platform. First off, second off, while they say they don’t use it for training data, there is no guarantee that it is not being stored somewhere else, and there’s also no guarantee that the the thing you tell it to do one time, it will continue to reproduce exactly in the same way because it is not. It is statistics. So statistically speaking, like when you have a normal standard deviation, right? You the most of the time it will do the thing right in the middle, right at the top of that peak, right? But it won’t always. It’s a scatter plot, and so there will be moments that it does things that you don’t expect it to do. And if it’s doing something with private data that you don’t expect it to do, that’s terrifying,
Donny Shimamoto 10:11
extremely terrifying. And that you just that explanation you just gave, I’ve never thought of it that way. But that that’s really insightful as well because it it is statistics and most people think well in statistics if it says like 99% of the time then it’s going to happen 99% of the time or most of the time but what people don’t realize but the 1% does exist and so you need to worry about that 1% especially in a case like this
Liz Mason 10:40
you do and it’s it, and not necessarily like worry. It’s more be aware, right? Understand the implications of what you’re doing and what the risks are, right? Like that’s what what we do for our clients is we say like, hey, you want to make this tax decision, right? Well, I feel that I probably can do that in court. There’s a good chance I can defend this tax position, but there’s a small chance that I can’t, and that you would lose, and you would owe X amount of money. So look at it from that perspective with yourself as well, and say, okay, I understand the risk. Like it’s a it’s a probability machine, right? So there is a risk. There’s a small risk, but it is a risk. I am willing to take that risk. I am unwilling to take that risk. What would it cost me if I did? And I, you know, equate it to people like, okay, so we don’t email tax returns. Why don’t we email tax returns, Donnie?
Donny Shimamoto 11:32
The risk of privacy. That there’ll be a data breach on it. Someone exactly.
Liz Mason 11:35
But is there a likelihood that there’s going to be a data breach that somebody’s going to get into my transferred email and pull that tax return.
Donny Shimamoto 11:45
Of course, I got to give you the cybersecurity answer, which is yes, and and it may not be our side; it may be the client side that actually has the risk, and then but we’re still responsible in a sense because we’ve we’ve contributed to that risk. But it
Liz Mason 11:57
what does it require? It requires a breach, right? It requires a breach on one side or the other in some malicious way. It is not that 100% of the time when I email a tax return to you, it’s going to be in the open internet. It is a risk that I am unwilling to take because I do know that there is a chance somebody could get into my email or yours, and that information would then be public. It is the same with AI, right? Like every time you upload a tax return, is it going to be at risk? It is now in a in a place where if there is a breach, if somebody wanted to access that information, they could, and it would be much easier than getting into your secure encrypted vaults where you store your tax returns, or much you know, easier than going into your client portal that are very deterministic, very encrypted, very locked down information. And so, understanding that there is a risk there, it might be a small risk, but it is a risk. It’s a cyber risk. Makes you think about it differently, right? So, number one, security. Make sure you have good cyber liability insurance. Like regardless across the board, that’s good advice. And you could do everything perfectly and still end up having to use that insurance. And then understand what your deductible is on it, right? Like if you have a policy and your deductible is $100,000, then your risk is $100,000. But if your deductible is like 5000, that’s much more reasonable, right?
Donny Shimamoto 13:21
Well, and I also tell people it’s not just that. That there’s also that risk that you got to notify all your clients that this breach potentially occurred, which could affect a lot of whether a lot of them are going to use you next time, yeah,
Liz Mason 13:33
absolutely. Um, and whether or not actual information was taken, right? If the breach occurred, but the breach on an AI system, it’s harder because it is not your particular vault; it is the AI company. And then there’s been hacks that have been really, really cool, in my opinion. Right? Like there was that whole hack where if you share any part of your your chat in within Claude with somebody else, it now creates a data packet that gets moved, and that data packet could be unencrypted by a lesser model without any safeguards on it. And so, there, it becomes easier when you start researching and understanding how the technology works and shares data across the board. And it’s much like stealing an email; like it’s it’s effectively the same function. It’s just in a different world,
Donny Shimamoto 14:22
and you’re actually reminding me too about there was an article that I read about how the AI to help communicate back here the issues that we found. It had taken screenshots of some of stuff which included PII, and it started in somewhere that wasn’t secure in in the in the code repository to then send that back, not realizing that act that actually now created a PII disclosure issue.
Liz Mason 14:48
Yep, yeah, it’s really interesting, and even like there’s software that we use to like monitor our team’s performance, right? So like some people put monitoring software on laptops that’ll take screenshots. Those screenshots could include tax data, and if that tax data is stored in their systems that are not built for regulated industries, what does that mean? That’s another potential breach, right? And it is a low risk activity because you know that ultimately, like that stuff is slightly encrypted, maybe not SOC two compliant, but it’s not not you know like it’s it’s not just throwing it out to the general internet, but understanding this in these terms is very important for people to start as a baseline. And I say that because I’ve talked to so many accountants this year. I’ve been at like 20 events, speaking and talking, and like interacting and understanding like where they’re at, what they’re thinking about, and people are not truly recognizing what the cyber liability risk side is and how it works, they’re saying no. It’s totally safe. It’s as if I put it into, you know, Carbon or another practice management system that is SOC two compliant and totally encrypted and GDPR compliant and all the things. But it is not. It is very different. And so understanding that it’s more like you’re emailing a tax return versus otherwise, like is very important. So now that I’ve scared everybody, can we talk about building? Because I am really excited about like the prospects of what you can do on the upside.
Donny Shimamoto 16:11
I do, but I just I just want to confirm one thing because it’s still sitting with me. Did I hear you right that you said even the enterprise subscription to Claude says you shouldn’t be providing storing. I’m giving Claude access to BII.
Liz Mason 16:25
Correct.
Donny Shimamoto 16:26
Whoa, because I know so that I mean that is the leading platform that I see firms that are trying to work with this work use. No,
Liz Mason 16:32
there are different ways to do it. You can self-host some of these models in a secure server environment. That’s a different story. But for the general purpose, like even if you’re paying for the you know team subscription on Claude, general purpose, do not put PII in it unless you would email a tax return.
Donny Shimamoto 16:52
Because that that’s not even just tax returns. That well, because a lot of tax firms do CAS work as well, right? That’s also CAS because you might be handling W 9s, you might be handling bank account information for payments and stuff, so that there’s actually a much broader risk to using Claude in that case. Well, and any any LLM, let’s qualify that any LLM where it’s going out, but specific Claude because Claude again is the one that I’m I’m hearing most people that are trying to do this stuff are using Claude. Wow! Yeah,
Liz Mason 17:21
wow.
Donny Shimamoto 17:22
Okay, let’s let’s transition. So, because you’re getting me scared now. So, if I want to build, how do I start building to with all of these guardrails in in mind?
Liz Mason 17:36
Yeah. So, also, I want to clarify on like the enterprise piece of it. There are ways to like set up an account with them where it is like HIPAA restricted or where it is like going to lock down completely. I’m talking about like the standard. If you just go sign up on their website, not very specifically or like creating a contract with Anthropic directly for special use purposes, but their general T’s and C’s say don’t do this. Like this is not a platform for this. Okay, so how are we building? So first off, considering safety, security is number one. So we have built like on the tax figure side, we built a privacy vault that like strips all PII out before any of the information touches an LLM, and that just gives the highest level of confidence that it can never potentially leak. And regardless of the fact that all the LLMs we’re using on the back end are highly secure, and also of that that higher level security, it does not it does not matter to me. I will not put any of our customers at risk ever. And the thing that we’ve done, which I think is like a really smart way to go about it, is because obviously we did is a council of models. And so when I say a council of models, it’s like okay, so somebody comes into the system and wants to look up IRA limits, or they want to look up depreciation rates, or something super basic that they would traditionally Google, which I want them to use our system because we have definitely better information than just the broad base of the internet, and you’ll never get a Reddit answer, TikTok tax advice, which is, I think, better for for us and particularly staff members that are up and coming in these firms need that restriction of information and not Google support. So, the Council of Models first off runs it through a flow at the top that says, “Okay, is this something like small lookup? Is this a big question? Like, how detailed is it?” And then it does prompt and contextual engineering, so it knows how accountants speak. We’ve trained it on the way that we talk about things, so that it can fill in all the context for the larger language model before it gets there, so that your answers come back more accurately. So that’s the first step. And if it’s a small question like the depreciation lookup IRA lookup, it’ll go to a very small self-hosted model that has that ability quickly. And so it it is a self-contained server hosted and says like, okay, here’s. All these look up things. I can pull that information for you very quickly, and then it goes through a review process. Everything goes through the review process, where every citation is double checked and made sure that it actually is being interpreted correctly from a different model before it comes back out. If it’s a complex question, like let’s say I’m doing a multi-state, multi-entity M and A with a bunch of different like possibilities of structure, and I’m asking this like multimodal modeling question, right? That’s going to go to a frontier model. It’s going to go to a big model, probably Opus 5.5 or Fable 5.1, and it’s going to go through that process where it has the most compute power, and then of course go through the review before it comes back out, and then it might even do a secondary process too. So it might use two different frontier models to get answers, and then come back with different responses, and then kind of debate it out through that review process as to what is right, or and if it’s the same. Obviously, that’s easy. If it’s not the same, then why? And like, what are the the different sections of that? So that obviously takes a little bit longer and a lot more compute power. And the reason that we structured it like that is twofold. One being, we want to keep our costs for our customers as low as possible.
Liz Mason 21:15
So if you are running every easy question through a high end model, you are using way more compute power than you need to, and so if you’re hearing like about the server farms and like all the you know environmental impact that people are concerned about with AI, utilizing the biggest model possible for baby question is abusive. Like it just flat out is abusive in this day and age, and understanding that doing that is going to use more compute power, it’s going to use more resources than necessary to get the answer is number one in learning how to build on AI. And so we’ve structured very specifically to have the lowest environmental impact, have the best cost effectiveness for our our client our customers, and also be thinking holistically about like what is the right type of model to get the most accurate, but limiting it. So it’s kind of like when you think in terms of who do you delegate it to? You delegate it to the cheapest resource, right? Like that’s the the standard business advice. It’s the same thing with AI models. You delegate it to the cheapest resource. Like you don’t want to be overexerting resources for this.
Dave Maresca 22:25
In today’s world, cyber threats don’t just target big businesses; they target you. Your small office or home office is at risk.
Alisa Nishimoto 22:33
There are a lot of solutions out there. Not all of them have the same level of security. The Soho Protection Package is designed to provide proactive cybersecurity solutions.
Dave Maresca 22:41
If your firm is looking for assistance in developing your personalized IT policy, connect with an advisor today at link.improvetheworld.net/sohocyber. Gain peace of mind that you have the right technology in place to stay protected.
Donny Shimamoto 22:57
Yeah, I love that. The sometimes the way I think of it is if you can do something with a shovel, you don’t need a big backhoe to to do it. And the the you just brought in the resources and the costing piece, which I also think that’s where we start to use this whole tokenomics kind of concepts of like if you give it to a smaller model, it might take say one token. If you give it to a big model to get the same answer is going to take like five or 10 tokens, and so there is this this trade off or cost efficiency that you’re going to look for in these. To take what you just said and translate that back to let’s say more like a citizen developer or the the non non product developer. So one one thing I heard you say is they may want to consider using a smaller model, or I’m also going to point towards some of the other vendor products allow like an API or something where you can go and inquiry their products. If you don’t know how to build the smaller model that’s going to have the authoritative guidance, tap into a vendor that provides that to you. But if you are, you can use one of these smaller models because those have already been proven out to look at this limited set of you. You gotta you gotta build it right and train it right, but you can build it off of the authoritative guidance and use that to do your lookups. Compared to then you use the bigger things and you need to have that prompt. And there was two things I think that you said that I always want to remind people about. One is providing the right context. So, what am I trying to achieve? And in some cases, how do you need to break this up to properly analyze this? And then the second one being the prompt that really the prompt itself. So, providing the context for the prompt for it to work with, and then the actual prompt for what you want it
Liz Mason 24:39
to do. Absolutely. And I think another thing, like another way to think about that, is like we don’t our clients come to us for tax advice, right? They come to us for tax advice because the tax code is 4 million words at the federal level, and there is no way for them to interpret all of that. There is no way for them to even understand all of the documents that they need to collect to get the. Answer. Like, think about how hard it is for them to be like just giving you tax documents at the end of the year, and then there’s always a like, oh, you sold a house. Can I please have like the HUD one? And they’re like, what’s a HUD one? Why would you need that? Like, they don’t even know the context to give to you. And so, if you think about that, think about an AI as if it’s a client, it needs all of this context, but it doesn’t know what context it needs. You do. You’re the professional. You have the expertise. You know the context that it needs to actually answer the question. So if you’re coming in asking like super baseline like questions without all the context, you will get a wrong answer, and that’s a another problem. So one of the reasons that we are selling tax figure directly to advisors is because advisors are the ones that have the context and understand it, and they are the right human in the loop. And the intention is empower the human to do more with less, utilizing really cool tech on the back end without having to build it themselves. Which of course, like you totally can build some of this stuff yourselves, depending on what you’re doing, and also understanding that like we will elevate our expertise, our experience, our humanity, our ethics, our empathy. All of that goes into that contextual information, and we need to be cognizant that there is no like even if our client sends us a Claude, hey, here’s my tax plan. They probably didn’t give all the context. They probably didn’t give all the information needed. They probably didn’t give accurate prior year information, like from what I’ve seen from my clients, anyways, right? And so then you get this like tax document, and you’re like, how do I even respond to this because it’s so wrong, right? And it’s not wrong because the AI is wrong. Like some, sometimes it is AI hallucinate a lot, but it’s generally wrong because the individual didn’t give it all the context or all the information to even understand the whole picture.
Donny Shimamoto 26:53
I’m laughing and I’m laughing in my head because just yesterday I was at a conference and one of the sessions was about the impact of AI, and and this was this was a farming conference, and so it wasn’t like like an accounting conference. But the guy that painted the picture, who’s like this guy formerly from MIT, and you know he worked on some major Silicon Valley stuff, and he was on the stage at Microsoft talking about about what AI is going to do, and he was like, “AI is going to automate 89% of the jobs.” And I went in IT, maybe not in the rest of what a lot of the rest of us do, because even and we were talking like 10 years out, and I went even 10 years out with whatever improvements there are. There’s it is there’s a lot that it’s not going to do, and you just gave all the reasons why it’s not going to do be able to do all that?
Liz Mason 27:43
Yeah, yeah. So I, you know, I think it’s like it’s a really fascinating moment to be in this space and building in this space. So tax figure was selected for the Battlefield 200 with TechCrunch at Disrupt, and so this week I spent the entire week at Disrupt, and I went to some really incredible sessions with very interesting like thought leaders in the tech space. Although they have a whole AI stage, just I was like, yeah, of course they do. Why would they not? But the I think probably the most interesting session I attended that was more new to me was a session on the impact environmentally and what people are doing to mitigate that. And so, you know, I was sitting; it was a roundtable discussion, and so there were actually 30 professionals involved in the discussion. And I was just like on the wall listening to this. And there were people from, you know, Nvidia. There were people from public utilities. There were people from like other large international data farm centers, there was a guy from Google. Like it was, it was very impressive. Like the caliber of people that were sitting at that table talking about this, and the theme across the board to every single person was, “How do we do this better, utilizing less resources?” There was no question. It wasn’t a “Do we need to consider this?” It was a “How do we do this? What are you doing? Let’s share best practices.” And you know, I think that that alone is really incredible to hear. You know, these major worldwide players coming to the table saying, “Let’s share best practices. Let’s collaborate. How do I talk to you after? Here are the things we’re considering. Instead of should we? And I think the media right now has portrayed tech, particularly AI, as very abusive and wanting to use more resources when it’s quite frankly the opposite. Like nobody wants to use more resources. In fact, like it’s a disincentive economically because they have to pay for more resources, right? And nobody wants that. And so, you know, I think it’s like it is fascinating to learn that this is a conversation from farm industry to high tech industry to accounting industry, and the ethical components of it are are coming to light across the board. And so, you know, I think there’s there’s been a lot of new developments. Like, have you played with Jev at all?
Donny Shimamoto 30:06
No, but I’ve I’ve started watching some of the videos about that, and yes, that looks that looks like a really great development path.
Liz Mason 30:14
Yeah, and so in basic terms for our accounting listeners, Jev is a tool that sits on the back end, where effectively you can use it to utilize way less of the AI compute to get to the same answer. And so there are people working in the the technology sector to build more of these types of models. That instead of iterating one after the other after the other, which is how a traditional language model works, it’s iterating in parallel in many different directions, so that it can do much more, much smaller iterations to come down to the the correct answer. And when you think about it from the perspective of mass impact, it can take big compute problems and truncate them down, and that that will just by default utilize fewer resources and effectively change the way that AI is being used. And the other thing I really liked is the founders of the company that produced Gev very explicitly on Twitter said, you know, they had a conversation with investors, being like, “Oh, you’re leaving money on the table. Like, you could charge more for this, like, because you’re saving them so much.” And they were like, “That’s not the intention. The intention is not to use more resources in any direction. It’s to make a decent business model and give this technology to as many people as possible, so that we can actually save economically.” And that’s a big shift in the way that that people are thinking about things in the tech industry, and that is very exciting to me.
Donny Shimamoto 31:43
That is, I think, I think you, Byron, and I need to do an arc episode on that one. That one definitely. Let’s let’s come back to this before we lose all the tax people that we’re talking to. So, you brought you brought up the ethics. So let’s kind of go there because that ethics and quality are actually two of the things that I’m also involved in at the international level, more kind of right now, we’re looking more at the audit side because of the impact of AI on on ad. But often it’s thought of as an audit issue. How does this play out from a as a tax ethics issue?
Liz Mason 32:17
Well, I mean, like what would happen? Like consider this with me. Everybody in one industry goes to Claude and they say, “How do I reduce my taxes?” And now you have an entire industry that is being encouraged to commit tax fraud because, as we established earlier, there is no chance they’re going to give it all of the context to come back with like the actual answer. Right? What does that do to our economy? We already have a $600 billion tax gap annually between what’s filed and what’s actually owed. So, like, how how do we as a country maintain if everybody across the board is being encouraged to commit tax fraud. This is a giant ethical issue, and it’s a conundrum, and probably like a pretty good update on philosophy, right? Like I should probably write some philosophical prompts out of this because it is happening in real time, and it is our job as tax professionals to be that ethical layer and say no, just because the internet thinks this might be defensible. My experience, my expertise says no, and here’s the context around that, right? And so I think at scale, if we release AI to the end user without our professional input, we have a giant ethical conundrum, and so it is up to each of us as tax professionals to like hold that close to our chest and ensure that we are reviewing and we’re discussing with our clients on a regular basis, like why this is ethical, why it is not, why it is truly possible, why it is not, and not just push the boundaries because potentially AI found a maybe loophole that we didn’t know existed before, or you know, is encouraging people to abuse the tax law. And then I think like empathy is another side of it, right? Like even though you can deduct a home office for years, if you know your client is about to sell their house, like that might not be the right answer for them, right? And you know, there’s always abuse in vehicles and businesses, which of course is being encouraged. And now, you know, all these TikTok influencers have told everyone they need to do the Augusta rule, but that only works in very specific instances. And we know that it is our ethical job to tell our clients that and to help make sure that we are preparing like actually accurate, defensible, and not abusive returns.
Donny Shimamoto 34:47
So, some of what you’re saying is like what from the internet, and what you said earlier though is that we should be using something that was more contained, so it didn’t have all that garbage in there. So. Say we are using something that is more contained for our work. Is there still an ethics issue there?
Liz Mason 35:06
I don’t think there’s an ethics issue to using AI to elevate what we do. I think there’s an ethics issue to using AI to replace what we do, and I say that because like even with the best harnesses inside of the models, where it’s saying you can’t go outside these confines, like these, this is your corpus of knowledge. Don’t use anything from Reddit ever, right? Like even if we do that, if we fail to provide all the context, it still might come back with something that’s wrong. So we always need a human to review it and to you know build that human relationship and understand the implications because we’re not going to get it from an AI like if even if we send a form to an end client you know like they’re not going to give us all the information right like we get so much more out of them by building that relationship and I can’t tell you how many times we did like a first year small business individual combination return, and we did it wrong because we didn’t have all the information, right? Like that happens all the time, and so then we had to go back and say, “Okay, well, you didn’t tell us about these rental properties that you have, or you didn’t tell us about this other business k1 that’s coming in.” And so, if you just put a layer of AI in between and never build that relationship to get that extra information, I believe that’s unethical because if the AI is replacing you and just doing 100% of the job, and you’re never looking at it as a human, yet you’re signing that tax return, that’s an ethics violation.
Donny Shimamoto 36:34
Ooh, that’s a that’s a powerful thought. The key thing that I think you raise there is is judgment, because that’s one. Even on the audit side, when we’re looking at it, you cannot use AI to replace judgment and that final review, which you’ve emphasized quite a bit on this. That even if it’s producing the work paper or the work product, that there needs to be a human review in this additional layer that says, “Does this make sense for this client?”
Liz Mason 37:01
Yes, absolutely. And like you know, I think holistically, yes, we can build agents to do a bunch of the prep work, the collection of documents, the compliance. And I think people should be doing that. They should be doing it in a secure way. They should be considering all of the other things we’ve discussed previously before going and building those agents, but it is totally possible, and you could easily go into, you know, ChatGPT or Claude and say, “Help me build my first agent. This is my intention. Here’s all the security things I’m worried about. Here are the systems I want to integrate with, and it will give you a plan and help you build it. And quite frankly, it’s not that difficult. I have encouraged people in my life from all walks of life, from a friend who is a pastor to a friend who is a journalist, to go build agents to do things like manual things that they’re doing, and they did it. And they kept being like, “Oh well, I’m going to call you if I have questions.” They never called because they didn’t need to because they asked the question of the system and the system said oh do it this way right but before you do that you need to understand the technology understand the risks understand the harnesses that you need specifically for regulated industry and for the profession you need to get those 7216 disclosures out to your clients and signed and then at the end of the day, take a deep breath and say, “Okay, where is the human in the loop? What is the process? What is the policy?” And all of that is required by the IRS, mind you. And so a lot of people don’t know that, so they’re building and not doing that, which of course creates a violation and a circular 230 violation as well. So it’s it creates all of these messes that people are like, oh, I didn’t even know about that, and it’s like, no, you didn’t. So do your homework, understand the implications across the board, and then go build cool stuff.
Donny Shimamoto 38:49
Okay, wait, you just introduced a circular 230 violation. Explain to us why that’s a circular, why there’s a circular 230 violation there.
Liz Mason 38:55
Yeah, so the IRS interpreted circular 230, which of course puts like the restrictions around tax firms on how you do like your work, right? It’s like the ethical. I think it’s kind of ethical, but also just foundational. This is what you have to do to comply with the IRS expectations. And if you are preparing a tax return in a in a way that is not inside of that, like approved in that way, then you’re outside Circular 230, and that’s a problem, right? And so, what like 7216 is addressed in Circular 230, so it’s you know an interpretation in and of itself. But that being said, you need to understand and read and and read the interpretation that the IRS did of 230 and 7216 before you’re doing these types of implementations, so that you know what I can do and what I can’t do. And 230 requires process and policy around preparation. And so, in that process and policy, you have to have a human in the loop anytime you’re using AI, and you have to have those disclosures if you’re using it in certain ways.
Donny Shimamoto 40:00
Thank you. That ton of of sense, and that goes back to I think you, Byron, and I have all in the past said start with process, understand the process, look at what you’re trying to do, and sometimes it’s throw away your current process and restart. But you have to have a process. It’s not just throw things into the black box and accept whatever comes out.
Liz Mason 40:20
Right, and I do want to encourage people. Like from a holistic perspective, I can tell you some of the things I’ve seen firms build. I saw a firm build a self-hosted LLM that effectively just combed through all of their files on a, I think it was weekly basis to say like this is all up to date, we’re good. I reorganized files this way. They had an inbox where it was like all new tax documents went to this one folder, and then the agent would sort them into the client folders for them and give them a report. I saw another one that was basically going through their practice management system and saying like, “Hey, here’s all of the open items that we’re waiting for from clients. Do you want me to email them? Right. So there’s like a bunch of this administrative task that you can you can automate really quickly. Dashboards is another one, just showing like, hey, here’s where we’re at with tax. Like, here’s the progress. Here’s what’s showing in the project management system. Like, this is an outlier because you know it doesn’t look right. All the information came in two months ago. Why is it not complete? Right, like highlighting those things so that it takes less of your your thought process. And I think that those administrative tasks right now are really great places to start from an automation perspective. Or even like, did we get all the did the client say we got all the documents? That doesn’t mean we got all the documents. So let’s go read all the documents and make sure they’re all there, right? That’s another big one, and then of course like saving and filing the finished returns. So I personally would probably not give an agent access to my actual tax return prep software at this moment unless you understand all of these safeguards and are doing it in a way that is very safe. If that’s the case, then go for it. I would probably have you know those saved down into a folder and then have the agent from the folder redistribute them. And if you’re doing you know self-hosted or small model situations, that becomes a lot safer and a lot easier. And you don’t need a frontier model to do that kind of stuff. Like that is a very easy programming thing. You can use some of these tiny models and host them and wall them off yourself.
Donny Shimamoto 42:30
Yeah, yeah. And this is where I’m going to mention it, not an endorsement necessarily, but this is one of the reasons why I do like the way Microsoft has implemented some of their stuff. I know a lot of people complain about that, but they do have it. They they do have the way that they design stuff. It stays inside the Microsoft ecosystem, and when it does utilize an external model, which could be a frontier model, there is a de-identification process that they apply when passing stuff over. Not always. You have to specifically you have to set it up. Set it, yeah, and but that is that is possible. So especially for smaller firms, I usually will steer them towards that because it’ll provide a bunch of the infrastructure that they need if they don’t know how to set that stuff up themselves.
Liz Mason 43:11
Absolutely. So I think holistically, building in tax right now is huge. We have in the tax industry some of the most outdated software, some of the most outdated processes, and at the end of the day, like there is still a lot of administrative burden on tax professionals. And sometimes we have like whole admin teams just to do these things, right? Like we don’t need an admin team to print documents anymore. Like you can have an agent print all your returns, or you can have a list of hey, here’s all the returns that need to be printed, and then it can go print them, right? And you just yes, you will need a physical person to collate unless you have one of those robots at TechCrunch. They had like humanoid robots walking around doing activities. That was fascinating. There was also a moment where there was a like dog robot, and then there was a real dog, and they were like facing off, and the real dog was like, and the the robot dog was like, what was the most adorable thing. But that’s a segue. My point is, soon enough, we are going to have physical humanoid robots to do this work for us, and that’ll be absolutely amazing. And I can’t wait for them to fold my laundry in this moment. After you print the return, you’re still going to need a human to like put it in an envelope and mail it, but you don’t need a human to go through and figure out and find the files and print them, right? Like you can build an agent for that.
Donny Shimamoto 44:29
And I do want to highlight one other thing you said about the the whole the whole concept of of administrative focusing on administrative because for the audit side guidance that we’re that we’re writing, which usually will trickle over into tax as tax starts to do their formal ethics and and and standards of practice type of things. We are we are specifically excluding from quality management administrative tasks. So if if you’re used for administrative tasks, it’s actually not going to fall on. Under the quality management standards on the audit side, and I would expect FACS to kind of follow the same thing eventually. It
Liz Mason 45:06
does still fall under the security requirements of the IRS. I just want to clarify that, but it is a different level of review that’s needed when you’re automating administrative tasks versus judgment tasks. And I would argue that we are not at a moment where judgment should be automated; it can be informed. So I think about AI as creating superpowers for advisors, right? Like you can have all of these ideas given to you and researched and formatted and built in a way that’s easy to review, and it can take you know four hours of tax planning down to half an hour of like reading and validating, and that’s amazing. Like that is absolutely incredibly amazing, and that’s honestly what like that’s why I built Taxfigure was to give the advisors superpowers so that we can build better human relationships.
Donny Shimamoto 45:55
100%, 100%, and that’s where to the quick one. If you’re gonna do this, you just heard Liz’s change from four hours down to a half an hour. This is why value pricing or fixed pricing on a lot of this stuff becomes very important because you have to get stuff in there. Otherwise, you’re going to lose a whole bunch of revenue, and this stuff is such high value for the client. Way more value than the compliance work that’s being done.
Liz Mason 46:18
For sure, this has been a fun conversation. Thank you, Donny. It
Donny Shimamoto 46:22
is no, thank you. And I, I mean, really, you shared so much that is should really help all of these small firms that are trying to do this stuff, and especially the 7216 compliance that and the and the 230 that I did not know. And so having them reaffirming that is like it’s in the IRS code. It’s not just something our profession is saying; the IRS is actually saying. Hopefully, that hits home to a lot of them. All right, everyone, that’s a wrap. So, again, we thank Liz for sharing all this with us, and we hope to see you on another episode of Innovation Insights.
Liz Mason 46:55
Thanks, everyone.
One Response to “Before Building with AI, Firms Need a Plan for Client Data | Innovation Insights”
ITC & Co. Team
Good point on human review. One thing firms often miss: the same data questions apply to anyone outside the firm who touches client files, whether that’s an AI tool or an outsourced team. Who can access which ledgers, is access logged, is anything downloaded to personal devices, and who signs off before output reaches the client? A firm that has answered those for its outsourcing arrangements is already halfway to an AI data policy.