Natural Language Processing or NLP is a branch of Computer Science that deals with giving computers the ability to understand text the same way human beings can. NLP is essential for companies that have large amounts of text or speech data that they want to efficiently analyze and understand. There are many use cases for which companies can use NLP to automate their tasks.
As the volume of text and audio data continues to grow exponentially, companies face the challenge of efficiently processing, analyzing, and understanding this vast amount of information. Manual processing of these tasks can quickly become impractical due to the sheer scale of the data, time constraints, cost implications, and the potential for human error.
We need to look at our target users, what are the difficulties they are facing with NLP tasks and how we can solve them through Spectrum. For this, I created a comprehensive value proposition chart explaining the functionalities users expect and what Spectrum can be designed to offer. These requirements come from similar platforms such as Amazon Comprehend, Google Cloud Natural Language, Microsoft Azure Cognitive Services and IBM Watson Natural Language Understanding.
Spectrum may have a lower accuracy for some tasks when compared to its competitors as it uses existing models over custom models but more than makes up for it by the more streamlined user experience it offers. The complexity to use the platform will be much lesser than the competitors, thus making it ideal for non-technical persons to learn and use the platform efficiently. This usability will be the main selling point of Spectrum.
When creating a new product, it is important to focus on the minimum set of features necessary to meet the initial user requirements. This approach allows us to release the product quickly as a Minimum Viable Product (MVP) for validation.
By prioritizing the essential functionalities that address the core user needs, we can launch the product sooner and gather valuable feedback from users. This early validation helps us understand if we are on the right track and allows us to iterate and improve the product based on real user insights.
Releasing the MVP promptly also gives us a competitive advantage by entering the market early and establishing a presence. It allows us to gather user feedback, validate assumptions, and make informed decisions for future feature development.
In the world of product development, an MVP often starts with a single standout feature that defines its core offering. However, in a competitive landscape where established players such as Amazon Comprehend, Google Cloud and Microsoft Azure boast diverse feature sets catering to similar use cases, it becomes imperative for our product to not only stand up to that competition but also emerge as a compelling alternative to existing solutions.
For this reason, Spectrum will encompass a range of powerful functionalities, extending beyond a singular feature, to provide users with a comprehensive suite of NLP automation capabilities. This approach empowers users to maximize their productivity, streamline their workflows, and gain deeper insights from their data, all within a single, integrated solution and will establish its competitiveness when compared to other existing solutions.
Standalone services are those that can be accessed on its own while Integrated Services are those that can only be accessed when choosing an existing Standalone Service. For example, Spectrum will be able to perform Named Entity Recognition on a document that is generated by Spectrum Transcribe but the feature cannot be accessed on its own.
Instead of making my own custom models from the limited data I have available for each functionality, using existing state-of-the-art models that have undergone an intense training and testing process is the best method to get good results in the immediacy. There are drawbacks associated with this approach such as not reliably being able to capture domain-specific nuances or the lack of customization but for an MVP, this approach works well as it is less time-consuming and more cost-efficient. As the Spectrum user base increases in size and Spectrum becomes a bigger company, these are some potential features that can be incorporated.
Spectrum is designed with key goals in mind for a hassle-free NLP process and to make it a competitive platform within this landscape.
Additionally, the flow of each functionality should remain largely the same for improved learnability and familiarity. Subtle differences are present in customization options and the data dashboard to cater to the unique requirements of each feature. In rare instances, the platform may generate translations or classifications that are not entirely accurate. To ensure precision and user satisfaction, the option for users to review and edit the output as needed is provided. This user-centric design approach enhances the overall usability and satisfaction of the platform, empowering users to harness the power of NLP with ease.
To enhance the efficiency of the flow design process, I will focus on each functionality of Spectrum individually. This will include both the model implementation and its associated high-fidelity prototype.
Given an audio file, Spectrum Transcribe accurately transcribes the audio content. While Speaker Diarization is not currently included in this implementation, it can be expanded upon in the future by incorporating libraries such as Kaldi. This functionality is based on Facebook's Wav2Vec2 Model.
Given a text, Spectrum Empathise determines the sentiment expressed in the text, categorizing it as positive, negative or neutral. This functionality utilizes Cardiff NLP's Twitter-Roberta-Base-Sentiment-Latest Model. Since this model was trained on 124M Tweets, the possibilities of the tweets belonging to different categories is very high and thus, this model can be reliable for general text but unreliable for specific use cases.
Given a text, Spectrum Condense generates a concise summary while preserving the most important information. Users have the option to customize the length of the summary within a specified range. This functionality is based on Facebook's Bart-Large-CNN Model.
Given a text, Spectrum Enquire will be able to answer questions that you have pertaining to the text. This functionality will be based on Deepsets's Deberta-v3-Large-Squad2 Model.
Given a text, Spectrum Translate translates the text into the user's desired target language. The current implementation showcases English to German translation, but it can be extended to support additional language pairs. This functionality utilizes University of Helsinki's Opus-Mt-En-De Model.
Given a text, Spectrum Classify will be able to classify it to the labels given by the user. This is a Zero-Shot Classification where the model is trained on a set of labeled examples but is then able to classify new examples from previously unseen classes. For this reason, the list of potential labels for the texts have to be given by the user. This implementation will be based on Facebook's Bart-Large-MNLI Model.
Given a text, Spectrum Recognize performs Named Entity Recognition (NER). It identifies entities belonging to one of the 18 predefined classes for NER. This implementation is based on Flair's NER-English-Ontonotes-Large.
We have navigated the intricacies of crafting an intuitive and user-centric experience for Spectrum, a SaaS platform designed to empower users with seamless, efficient interactions. All the workflows have been carefully designed to meet diverse user needs, gathered based on existing products. The user interface prioritizes simplicity, ensuring a smooth and guided user journey.
Spectrum will evolve with the users based on regular feedback loops, usability testing, and data analysis as these are integral to the iterative process.
In this ongoing narrative, Spectrum should be viewed as unfinished, not in terms of incompleteness, but as a commitment to continuous enhancement.
In case you want to contact me for further projects, you can reach out to me at pradhyumnaag30@gmail.com.