Dongyeop Kang Developed AI-Based Text Editing System for High-Level Writing
Department of Computer Science & Engineering assistant professor Dongyeop Kang introduced a refined artificial intelligence (AI) based text editing system to provide writing assistance. In collaboration with researchers at Grammarly, CoEdIT aims to assist in more complicated editing tasks by providing an interactive collaboration experience.
“When someone is writing a draft, the grammar and spelling prediction is helpful, but it is not improving the quality of the final product,” Kang said. “When someone is writing something high-quality, they make edits as they go and polish their phrasing and structure. We wanted to build off of this more complex editing process. Now our model can revise a draft and we measure whether or not the edits improve the readability and reader comprehension. We then started developing a system that would be more interactive with the writer as they are writing.”
In the original version of their model, users get machine initiated suggestions and have to decide whether to take that suggestion or not. The users did not have the option to make requests or add more details about the type of suggestions they would want. Kang is working to make that process more collaborative and improve the model so it can understand the semantics of user’s requests and suggestions, like making a long sentence more succinct, or changing the language to be more formal.
“The goal is to make the model more human driven and improve the collaboration between the AI and the user,” Kang said. “So far, this model out performs many of the benchmark datasets from ChatGPT and GPT4. We have more than 10,000 downloads in the hugging phase - everything is open source so we measure the impact of the technology based on how many people are applying the model to their own projects and work.”
Moving forward, Kang hopes the model can begin to identify and understand the different steps in the human writing process and make suggestions accordingly. Additionally, his lab is working on understanding how human writing is fundamentally different from generative AI language. As more and more content online is written by AI, this technology could help identify AI-produced works and assist in copyright-related cases in the future.
“AI reasoning has improved greatly since I started this work and started to close that gap,” Kang said. “Planning is an area that has been under explored and needs more work to perfect. As for linguistics styles, humans base their language off the person they are talking to and the interpersonal context for any situation. Models don’t have that understanding. I am working to make models more socially grounded, interpersonal, and people centric.”
Kang’s lab is also working to improve AI’s abilities in “expert” spaces. Large language models like ChatGPT are generalists with the ability to do a number of things, but fall short in areas that require advanced expertise. By training in specific domains, AI models may be able to assist humans in more complicated tasks in the future. Kang is working with Open Philanthropy, which is a nonprofit organization with a focus on creating AI benchmarks, to start tackling this project.
“If we want AI to be able to assist with more complex tasks like legal, medical, and research work, it needs to have expert knowledge and abilities,” Kang said. “We proposed an ambitious project to do in-depth interviews with experts in different domains to create a new benchmark to collect expert knowledge and skill sets to assess the current gaps in our current AI systems. We are starting with the legal domain and have done interviews with lawyers to understand their day-to-day work.”