Table of contents
- Foreword
- Feedback as an important factor in the relationship between management and employees
- The same goes for intelligence systems
- What is feedback
- How to give feedback
- When to give feedback
- Why give feedback
- Go Data-Centric for Production Sustainability
- Summary
Foreword
As CEO of an AI company , I always find myself at the intersection of technology and talent. One cannot operate without the other. Being at the forefront of evolving artificial intelligence paradigms, I often go back and compare and find correlations with human intelligence. This time, I would like to think about feedback, which is one of the correlations, and how to approach employees and how to explain the complex process of AI feedback in a friendly and simple way.
Feedback as an important factor in the relationship between management and employees
When considering personal growth, feedback is always talked about as an essential element in the process. Feedback is a fundamental mechanism by which we learn and understand how others view us and our work. It’s very easy to understand what feedback is, but hard to master. I imagine a good manager to be someone who can communicate his thoughts in a straightforward manner without squeezing my ego or blaming me for my mistakes, but saying it clearly. Such a person
A common challenge in the tech industry in 2022 is workforce attrition and engagement. Feedback, both as a tool and as a value, is a very important aspect to adopt as it energizes the entire organization to address this issue. But many experienced managers I’ve met in recent months struggle with giving direct feedback. Building a culture of “radical candor” (see Kim Scott’s book and website ) proved to be no easy task.
Good feedback, even candid, requires building a strong foundation of trust that is normative, not something that happens quarterly and arbitrarily. Some people call this feedback “continuous feedback,” while others focus on being open with the people they work with in every meeting and internal communication. What’s clear is that working without proper feedback doesn’t work, so it’s necessary to close the feedback loop.
After completing my PhD in AI, and as a young executive wanting to learn more about feedback in general, and feedback in a startup environment in particular, I started reading every book I could on the subject. I’ve come across many fascinating approaches and perspectives, but my thinking has been to focus on asking the right questions:
What is Feedback?
how to give feedback
when to give feedback
why give feedback
I have found that these questions are not easy to answer, and that the answers vary by organization, manager, and employee. However, they all have one thing in common: feedback is important to humans. In a world where personal growth is so important to employees and professional development determines employee satisfaction, it’s no surprise that feedback is so important.
Regarding management theory based on radical candor, there is a Japanese translation of the book, “ GREAT BOSS – Silicon Valley Style Outspoken Power ” (published by Toyo Keizai Inc., March 2019). Also, Newsweek Japan published an article on the theory on August 8, 2017 .
The same goes for intelligence systems
After spending a good amount of time exploring the idea of employee feedback, I went back to technology, especially AI/ML models. AI/ML models, like humans, need feedback to improve. Information about model performance is essential for model training. Leveraging such (performance) knowledge on the model can better accomplish the problem of generalization.
Let’s imagine a simple scenario (regarding the task of initial construction of a new model) of training a basic model with 1000 data samples. Let’s say the model you just trained was at version 1.0 and then implemented in production and processed the data to predict something. For example, a model could be trained (using data from a CRM system) to predict whether a sales opportunity will close.
After a few weeks, some sales opportunities were converted and others were lost. As expected, the model was correct in most cases, but wrong in some cases. Closing the feedback loop means using new data (not used in training). Specifically, it captures where the model went wrong and uses the data to improve the first model, version 1.0. Instead of a 1000-sample dataset, we can train a version 2.0 model with 1000+100 additional samples.
It may sound simple, but having such a process in an AI system is by no means an easy task and requires difficult development. The four questions mentioned earlier in the context of employee feedback are also relevant in the context of AI systems.
What is feedback
This question is not an easy one to answer. My short answer to this question is that feedback is new labeled data that can be used to train a model. how to sample how to annotate What is sufficient feedback data? The answers to these further questions will vary.
How to give feedback
In good processes, the responsibility for feedback is given to domain experts, who are usually not technical experts. Therefore, the method of feedback (performed by domain experts) should be determined collaboratively by the data science team and the business unit. The presence of noisy data poses a major challenge in the model training process, so it is important to clarify the definition of annotations.
When to give feedback
A good rule of thumb is to give feedback frequently and regularly. Also, if the data changes or there is a drift such as a shift in the distribution, we need to start retraining the model. Basic monitoring (for AI systems) raises alerts to help you know specifically WHEN feedback is needed.
Why give feedback
The answer is quite clear. Feedback is provided to ensure that the model is relevant and effective over time. Not only does it improve the model, but it keeps it well in an ever-changing environment.
By laying the foundation for a feedback mechanism, companies can “get the model off the ground” and perform tedious maintenance activities more effectively. By doing so, you can reduce your total cost of ownership (TCO) and increase your return on investment (ROI) with AI across your organization.
Go Data -Centric for Production Sustainability
Closing the loop between users and (AI) models is all about turning feedback into action. This means using key information from the field to boost the system by focusing on new data. Something magical happens as a byproduct of this process. Production environments become data-centric.
How is that possible? With proper feedback mechanisms, simple models can be put into production and improved with better data from operational processes. This constant feedback and growth process saves research time and money that would otherwise be invested in creating an ideal and robust model. Just as there are no perfect employees the first time, there are no perfect models the first time.
AI system development .
|
Summary
From employees to AI models, feedback is a powerful tool that drives business forward. On a human level, mastering it and building the right feedback processes empowers people across the organization. At the level of digital transformation, this element of user experience drives the productivity journey, resulting in broader adoption, greater scaling, and better ROI.
