SHIFT: Testing and QA PMO Service Case StudyInterview with the Testing and Development Departments for Clip Studio Paint
Testing and Development Departments working together as one
Quality assurance for large scale projects and breaking away from office divisions
Support with software engineering, vast know-how, and tight-knit capability
Summary
CELSYS, Inc., the developers of Clip Studio Paint, the illustration, comic, webtoon, and animation app loved by graphic creators around the world. The test volume for the app grows with a total of over 30 million units provided worldwide and its availability on many OS and devices. To address that, the quality PMO service SHIFT helped to streamline testing and reexamine the upstream development process.
We didn’t just request them to help with the testing workload.
We wanted to improve ourselves while receiving their support.
- Please share the challenges you faced and the situation that led to bringing in SHIFT’s support.
Product Support Department Manager Shimada:
As Clip Studio Paint develops over the years, so does the test volume, so we considered automation to review test streamlining and organization, as well as the workflow. At that time, we consulted several debugging companies and found SHIFT. The software testing service grabbed our attention and that is what got us to choose them.
The testers at SHIFT are a group of professionals who passed an entrance exam, which makes us feel secure. The structure of having separate project managers and designers, and testing what exactly what we wanted to do according to a standard viewpoint sheet, were very appealing to us. Also, as Clip Studio Paint is a rather unique app, we have to start with detailed information and task input when requesting services outside our company. However, SHIFT had a track record for similar cases, so we requested their help thinking that we could not only improve our test quality, but also learn a lot from them.
- What was it actually like working together?
Shimada:
Differing from when we previously outsourced companies, which only worked on their project from start to finish, we received relevant feedback after every test finished, as well as a set of issues and suggestions. We are very thankful for the great learning experience, and for the issues and awareness from an objective perspective that we would not have realized on our own.
Also, just by working with the mindset like a Celsys employee, the two people in charge of the project are very reliable and trustworthy. For example, they catch up and give suggestions and consultations beyond the set scope.
- That’s great to hear.
After that, I heard that you introduced SHIFT not only for testing, but also crossing department borders to the Development Department.
Shimada:
That’s right. While they provided actual testing support, I felt that there were some issues in the upstream process that couldn’t be solved by just improving the skills of the Product Support Department. During that time, we received a proposal for solving issues related to the upstream process, and so I discussed it over with Inaba of the Development Department, thinking that it would be possible to approach it together with SHIFT.
Their strengths include software engineering and multiple development site know-how, as well as tight-knit on-site capability.
Building a development system that supports large scale projects together with SHIFT.
- What kind of issues did the Development Department have when Shimada approached you?
Director and CTO Inaba:
I believe it was around October 2022 when SHIFT approached me with the suggestion of a quality PMO service to review the whole development process. At our company, our development style is to have a developer be in charge of one project, and with the discretion of the managers and other developers, have them see it through to the end. However, as the scale of the product grew and the number of people and departments involved increased, we felt that there were limits to this approach.
With these circumstances, there were three major goals that I wanted to achieve with SHIFT.
1. Establish a development process that clearly documented the upstream to downstream process.
2. Create standard documents to reduce rework and smoothly respond to questions from the testing department.
3. Set up and learn the know-how to create test design within our testing department.
I believe writing down these details in a memo and handing it to them was the start of the communication between the Development Department and SHIFT.
- What were the appealing points about SHIFT that made you decide to ask for their service?
Inaba:
Before receiving their service proposal, I took SHIFT’s Hinshitsu University* course. I originally studied software engineering in university, so I knew the general workflow for the development process, but I learned a lot from taking their course. While it had a solid software engineering as a base, the course content was reinforced with know-how and case studies from various projects in which SHIFT had been involved in, so I had high expectations for SHIFT.
One other appealing point was their tight-knit capability.
For example, even with correct software engineering knowledge and when told what’s correct, applying it on-site is another story. Especially for our company, despite having its issues our app has established a defined position, so when changing that development structure we had apprehension. We asked ourselves would things actually be improved? Or rather, would the things that are going well start to go wrong?
However, what really appealed to me was that, in addition to the notes I shared, I received surveys with those who were already involved in the testing department, and suggestions that understood our company's in-house production mindset, etc. which were tailored to our needs.
With SHIFT, we could improve our development to a higher level with the general software engineering theory as a base. As I had the confidence and high expectations, I decided to request SHIFT’s help without hesitation.
*Hinshitsu University is a specialized education institution that puts into words and systemizes the knowledge of Software Testing and Quality Assurance that SHIFT has cultivated up until now.
- Thank you. What was it like working together with them in the Development Department?
Inaba:
Over a period of four months, they analyzed our current development process, and with that in mind, they changed the operation work flow for various tasks and improved our documents, as well as set up our test design structure. Implementing the new operation work flow on-site has yet to happen, but if applied to past projects, around 30% of tasks during development can be reduced in advance.
Regarding using the new operation work flow and standard documents, before the trial some development members felt that it would be difficult to implement, but they found that it was surprisingly faster to use the new documents. Many commented how it made work easier later on and they have hopes for future applications. We also received a request from the Testing Department that they want 100% implementation! (laughs)
Shimada:
As a testing department, even if there is a cost to implement it, we are looking forward to documents being shared in a proper format, obtaining that information, and being able to design tests ourselves.
Inaba:
There was time left at the end of one of the meetings, and O-san from SHIFT gave us an analysis of the issues they noticed while looking at our development. It was completely extra, but it was an analysis with observations that I had as well, such as we’re losing out by not having this. The Development Department was very busy at that time and I haven’t been able to share it within the department yet, but I would like to use a bit of it.
Shimada:
I thought that O-san really made a great point and I immediately shared this within my department.
Inaba:
You were quick to share it! (laughs)
Many engineering specialists are in our company, and on top of being particular to our in-house production, many have successfully done their work independently and haven’t really thought about collaborating outside the department, so it was good that we asked for SHIFT’s help. It also became easier internally to suggest doing something a certain way, because a specialist with experience with so many companies said we should.
Going beyond making the quality and structure to meet the demands of the expanding user base
- Please share what Celsys is aiming for in the future.
Inaba:
This time, SHIFT made a workflow and structure for us that would be difficult to integrate immediately within our company. I thought of introducing a system that would be practical and easy to apply at first, but if we’re going to improve we might as well go for a higher level. We are planning to try out a few selected matters while gradually increasing the number of developers who can work on the new development workflow, and integrate it over the next year or so. The reason why we were able to build a new workflow and structure was because Shimada introduced SHIFT to us. I’m very grateful.
Shimada:
Development and testing go hand in hand, so proceeding with the same goals and awareness will lead to higher quality service. I think that the problems we have while testing are troubling the development side at the same time, so the biggest thing was the understanding and acceptance of this endeavor by the Development Department. I would like us to cooperate with one another and further improve our quality smoothly.
Inaba:
There is truly no limit to what we want to do at Celsys. I would like to use this opportunity of requesting SHIFT’s help this time to set up a structure that could support large-scale products over a long period of time, improve quality, increase what we can do, and provide even more to the creators that use our app.
Note: This article was originally published as a Case Study on SHIFT in August 2023.
