Shahab Khan

Work

Work

About Me

About Me

Resume

Shahab Khan

Fixing API documentation with Anomify

Fixing API documentation with Anomify

Solo, Developer Experience, DevRel

Solo, Developer Experience, DevRel

A project undertaken for Anomify - A B2B company which offers AI-Driven advanced data monitoring solution. The Study focused on optimizing the API Documentation to facilitate better Understanding and better perceptions of Trust and Brand Value

A project undertaken for Anomify - A B2B company which offers AI-Driven advanced data monitoring solution. The Study focused on optimizing the API Documentation to facilitate better Understanding and better perceptions of Trust and Brand Value

Client

Client

Anomify

Anomify

Services

Services

Primary Research

Secondary Research

Market Analysis

Usability Testing

Prototyping

Validation Testing

Primary Research

Secondary Research

Market Analysis

Usability Testing

Prototyping

Validation Testing

Industries

Industries

IT , Technology

IT , Technology

Date

Date

September 2023

September 2023

Overview

Overview

Anomify is a SaaS product designed for companies that need to react quickly to anomalies in their systems and monitor the health of their systems.


Anomify’s service requires clients to send their data in the form of a time-series at regular intervals.


The process of sending this data is typically performed by developers working for their clients in their operations or engineering teams.

Anomify is a SaaS product designed for companies that need to react quickly to anomalies in their systems and monitor the health of their systems.


Anomify’s service requires clients to send their data in the form of a time-series at regular intervals.


The process of sending this data is typically performed by developers working for their clients in their operations or engineering teams.

Problem

Problem

Developers must learn how to send data to Anomify via HTTP API. API can be defined as a set of protocols and requirements, that enable two different programs to send/receive data between each other.


It is critical that these requirements are conveyed in an effective and simplified manner to enable easy implementation by the developers. This is where the difficulty occurs, and this is where developers may be getting frustrated as developers might be used to reading different types of documentation for different software tech stacks

Developers must learn how to send data to Anomify via HTTP API. API can be defined as a set of protocols and requirements, that enable two different programs to send/receive data between each other.


It is critical that these requirements are conveyed in an effective and simplified manner to enable easy implementation by the developers. This is where the difficulty occurs, and this is where developers may be getting frustrated as developers might be used to reading different types of documentation for different software tech stacks

Design Process

Design Process

Due to limited time constraints and access to participants, A Double Diamond Process was used for it's flexibility. It consisted of the following phases

Due to limited time constraints and access to participants, A Double Diamond Process was used for it's flexibility. It consisted of the following phases

What Experts Say

What Experts Say

Secondary Research consisting of Academic Literature Reviews were conducted.

The Literature Review found that the most severe obstacles faced by the developers while learning and implementing new APIs consisted of learning content such as documentation.

(Uddin and Robillard, 2015)

Secondary Research consisting of Academic Literature Reviews were conducted.

The Literature Review found that the most severe obstacles faced by the developers while learning and implementing new APIs consisted of learning content such as documentation.

(Uddin and Robillard, 2015)

Competitive Review

Competitive Review

We see that the features and characteristics of software documentation that has been shown to significantly improve developer learning in the literature review has been adopted by developer first / developer plus companies such as.

We see that the features and characteristics of software documentation that has been shown to significantly improve developer learning in the literature review has been adopted by developer first / developer plus companies such as.

Hypothesis

Hypothesis

The Documentation negatively affects developer understanding, trust and value perception of the Brand.

The Documentation negatively affects developer understanding, trust and value perception of the Brand.

Primary Research

Primary Research

A Usability Test followed by a Questionnaire and Interview was carried out to answer the following questions derived from the hypothesis

1. What is the attitude or the impression that users have of Anomify while reading the documentation?

2. Are there any problems in the documentation that affect user’s experience in a negative way? If yes, what are they? if no, what part of the process is causing problems?

3. Are there any functional problems from Anomify’s API Service that prevents users from sending data?

A Usability Test followed by a Questionnaire and Interview was carried out to answer the following questions derived from the hypothesis

1. What is the attitude or the impression that users have of Anomify while reading the documentation?

2. Are there any problems in the documentation that affect user’s experience in a negative way? If yes, what are they? if no, what part of the process is causing problems?

3. Are there any functional problems from Anomify’s API Service that prevents users from sending data?

Target Audience

Target Audience

The users that face the problem hypothesized are primarily developers.

They like to learn by figuring it out and do not like being marketed to, rather, they prefer to build their impressions from data and facts.

Developers can play a vital role in the adoption of a third-party service. These roles can be divided into 4 categories:

1. Initiator
2. Technical Decision Maker
3. Business Decision Maker
4. Influencer
5. Approver

The users that face the problem hypothesized are primarily developers.

They like to learn by figuring it out and do not like being marketed to, rather, they prefer to build their impressions from data and facts.

Developers can play a vital role in the adoption of a third-party service. These roles can be divided into 4 categories:

1. Initiator
2. Technical Decision Maker
3. Business Decision Maker
4. Influencer
5. Approver

Qualitative Results

Qualitative Results

The interview and the usability study found core issues that resulted in a negative experience and an inability to learn how to send metrics. These issues can be grouped under the following categories

The interview and the usability study found core issues that resulted in a negative experience and an inability to learn how to send metrics. These issues can be grouped under the following categories

Different Documentation Pages

Different Documentation Pages

Incorrect and Incomplete

Incorrect and Incomplete

Too Much Information

Too Much Information

Navigation Index

Navigation Index

Quantitative Results

Quantitative Results

The Questionnaire’s enabled conversion of qualitative properties like trust and brand value into quantitative results which could be used to compare the participant’s experience between stripe and Anomify and set a baseline for future documentation prototypes.
This questionnaire involved a 10-point scale where 10 would be the highest score and 1 would be the least.

A modified System Usability scale was also used to visualize how developers felt about the usability of the documentation

The Questionnaire’s enabled conversion of qualitative properties like trust and brand value into quantitative results which could be used to compare the participant’s experience between stripe and Anomify and set a baseline for future documentation prototypes.
This questionnaire involved a 10-point scale where 10 would be the highest score and 1 would be the least.

A modified System Usability scale was also used to visualize how developers felt about the usability of the documentation

Prototypes

Prototypes

" The solution of a design problem is a negotiation of a constantly changing constraint field "

High Fidelity prototypes were created using the same markup language (MKDocs) that the developers of Anomify used to create their documentation. This was chosen over Figma or Sketch because:

1. Learning MKDocs would bring light to feasibility and constraints of what can be designed.

2. It would also allow high fidelity prototypes to be shared with developers easily who could then host it for validation testing in a matter of minutes.

3. Because of my background in Software Engineering , creating functional iterations and prototypes was extremely fast and easy compared to the usual pipeline of sharing design files with developers, who would then need to convert it to markup code.

" The solution of a design problem is a negotiation of a constantly changing constraint field "

High Fidelity prototypes were created using the same markup language (MKDocs) that the developers of Anomify used to create their documentation. This was chosen over Figma or Sketch because:

1. Learning MKDocs would bring light to feasibility and constraints of what can be designed.

2. It would also allow high fidelity prototypes to be shared with developers easily who could then host it for validation testing in a matter of minutes.

3. Because of my background in Software Engineering , creating functional iterations and prototypes was extremely fast and easy compared to the usual pipeline of sharing design files with developers, who would then need to convert it to markup code.

Unifying the different documentation Pages

Unifying the different documentation Pages

Support Docs and API Docs were combined into one documentation, but each represented a different section. Support docs was represented by “Getting Started” whereas “API Docs” was represented by API Resources. Taxonomoy of the API Docs navigation was also changed to make the API resources navigation more understandable

Support Docs and API Docs were combined into one documentation, but each represented a different section. Support docs was represented by “Getting Started” whereas “API Docs” was represented by API Resources. Taxonomoy of the API Docs navigation was also changed to make the API resources navigation more understandable

Defining a clear structure for API documentation

Defining a clear structure for API documentation

A clear and consistent structure was defined for all API documentations so that developers could familiarize themselves with the documentation and know where to find the required information.

A clear and consistent structure was defined for all API documentations so that developers could familiarize themselves with the documentation and know where to find the required information.

Highlighting important API characteristics

Highlighting important API characteristics

The documentation was designed to be easier to scan through by highlighting important terms and API characteristics

The documentation was designed to be easier to scan through by highlighting important terms and API characteristics

Improving Code Examples

Improving Code Examples

Adding Use Cases

Adding Use Cases

Condensing information into Tables

Condensing information into Tables

Adding Hyperlinks

Adding Hyperlinks

Validation Testing

Validation Testing

The Same User Tests that were carried out in the define phase were used in the validation testing with a new set of participants. The prototype provides a solution to the problems identified in the usability testing. However, there are some issues with the prototype that were found which must be addressed

The Same User Tests that were carried out in the define phase were used in the validation testing with a new set of participants. The prototype provides a solution to the problems identified in the usability testing. However, there are some issues with the prototype that were found which must be addressed

Easier to Understand

Easier to Understand

Useful Features

Useful Features

Better impressions of trust and brand value

Better impressions of trust and brand value

MKDocs Navigation problem

MKDocs Navigation problem

Issues finding the right documentation page

Issues finding the right documentation page

Participants got confused with the taxonomy

Participants got confused with the taxonomy

Quantitative Results

Quantitative Results

There is a significant increase in the ease of understanding and likeliness of recommendation in the prototype when compared to Anomify’s old documentation. We also see a significant increase in perception of value of the product in the prototype when compared to the original documentation. However, we see a small increase in the perception of trust in the product albeit there is an increase in the perception of trust in the product.

The System Usability Scale(SUS) results of the prototype shows improvements in all measures when compared to the SUS result of the original documentation. All participants agree that there was no inconsistency in the documentation. A majority of the participants agreed that they found the process of metrics easy after reading the documentation and that they were confident in their ability to send metrics. A majority of the participants also agreed that they would not need external help to implement the API after reading the documentation

There is a significant increase in the ease of understanding and likeliness of recommendation in the prototype when compared to Anomify’s old documentation. We also see a significant increase in perception of value of the product in the prototype when compared to the original documentation. However, we see a small increase in the perception of trust in the product albeit there is an increase in the perception of trust in the product.

The System Usability Scale(SUS) results of the prototype shows improvements in all measures when compared to the SUS result of the original documentation. All participants agree that there was no inconsistency in the documentation. A majority of the participants agreed that they found the process of metrics easy after reading the documentation and that they were confident in their ability to send metrics. A majority of the participants also agreed that they would not need external help to implement the API after reading the documentation

Key Takeaways , Future Scope and Limitations

Key Takeaways , Future Scope and Limitations

This study has found and solved some issues in the documentation which prevented developers from learning how to send data to Anomify’s service. It measured the effect that these issues had on developer understanding, trust and value perception of the brand and improved them via a prototype which was validated via user testing.

Developer Trust showed only a slight increase between Anomify’s original documentation and the prototype. As uncovered from the interviews, documentation could have a lesser effect on trust when compared to the company reputation. This requires further investigation as it could reveal significant findings pertaining to the problem.

This study tests the understanding of the API documentation rather than the implementation of the product. Testing how a developer implements an API after reading the software documentation of a product would require more resources than what is possible in the scope of this study. However, it would give deeper insight into how documentation affects implementation and adoption of an API.

This study has found and solved some issues in the documentation which prevented developers from learning how to send data to Anomify’s service. It measured the effect that these issues had on developer understanding, trust and value perception of the brand and improved them via a prototype which was validated via user testing.

Developer Trust showed only a slight increase between Anomify’s original documentation and the prototype. As uncovered from the interviews, documentation could have a lesser effect on trust when compared to the company reputation. This requires further investigation as it could reveal significant findings pertaining to the problem.

This study tests the understanding of the API documentation rather than the implementation of the product. Testing how a developer implements an API after reading the software documentation of a product would require more resources than what is possible in the scope of this study. However, it would give deeper insight into how documentation affects implementation and adoption of an API.