Difference between revisions of "OpenAPI development guidelines"

From MECwiki
Jump to: navigation, search
(Created page with "OpenAPI development guidelines for ETSI ISG MEC. More general recommendations can be found at [https://forge.etsi.org/wiki/index.php/OpenAPI_Guidelines Forge OpenAPI Guidelines].")
 
 
(16 intermediate revisions by 3 users not shown)
Line 1: Line 1:
OpenAPI development guidelines for ETSI ISG MEC.
+
OpenAPI Specification (OAS) development guidelines for ETSI MEC ISG API specifications.
More general recommendations can be found at [https://forge.etsi.org/wiki/index.php/OpenAPI_Guidelines Forge OpenAPI Guidelines].
+
 
 +
''(More general recommendations can be found at [https://forge.etsi.org/wiki/index.php/Main_Page Forge OpenAPI Guidelines])''
 +
 
 +
= Latest versions =
 +
 
 +
Latest versions of the OAS representations are available in their respective repositories ([https://forge.etsi.org/rep/mec here])
 +
 
 +
Latest CI test results of the OAS representations are available [https://forge.etsi.org/jenkins/view/all/job/MEC%20-%20Multi-access%20Edge%20Computing/ here]
 +
 
 +
= Contributing =
 +
 
 +
If you are willing to contribute or if you want more information, contact [mailto:cti_support@etsi.org?subject=MEC%20openapis%20development: ETSI CTI].
 +
 
 +
ETSI CTI has also made a [https://www.youtube.com/watch?v=YXO0R61Dcg8 YouTube] tutorial available (based on NFV) that provides guidance on how to contribute for MEC.
 +
 
 +
= API development =
 +
 
 +
== General guidelines ==
 +
 
 +
As of January 2021, OAS representations have been updated to reflect the latest released versions of their corresponding GS (e.g. v2.1.x);  the representations follow [https://swagger.io/specification/ OpenAPI 3.0 specification] language.
 +
 
 +
The following figure provides a high level overview of the OAS creation and usage.
 +
[[File:ToolchainOverview.png|center|thumb|500x500px]]
 +
 
 +
As shown on the figure, OAS YAML file is used in 3 different contexts:
 +
# To accurately reflect the GS API as released by ETSI MEC ISG
 +
# To provide graphical UI documentation as an input to SwaggerUI JS Application
 +
# To provide out-of-the-box API client library/package or server stub as an input to code generation tools
 +
 
 +
As such, when developing or updating the OAS YAML representations, the 3 usage described above should be considered.
 +
 
 +
The following sections provide guidelines on how to develop and validate OAS represnetaions.
 +
 
 +
== OAS YAML Generation ==
 +
Initial OAS generation involves using the ETSI maintained doc2oas tool; the figure below shows the steps involved in using this tool.
 +
 
 +
The following figure outlines doc2oas usage.
 +
[[File:Doc2oasToolchain.png|center|thumb]]
 +
 
 +
Step 1 - Create <code>MEC-XYZ_config.yaml</code> file
 +
* this file contains all external types not defined in the GS in (OAS language)
 +
 
 +
* existing config files can be used as a template, types can be re-used from existing config files
 +
* to quickly identify external types, Step2 can be performed with an empty config file - the resulting yaml file will show all unresolved types in the Swagger Editor
 +
 
 +
Step 2 - Use the <code>doc2oas.sh</code> script to generate <code>MEC-XYZ.yaml</code>
 +
* the new specification must first be added to <code>doc2oas.sh</code> script
 +
 
 +
Step 3 - Add endpoints as per the GS specification
 +
* follow the model from existing specifications
 +
* the end result should not show any error in Swagger Editor
 +
 
 +
== OAS YAML Validation ==
 +
Once the YAML file is available, the following validation steps should be performed to verify the validity in the different usage contexts
 +
 
 +
=== Level 1 Validation - OAS syntax ===
 +
{| class="wikitable"
 +
|+
 +
!Validation Tool
 +
!Validation Criteria
 +
|-
 +
|Swagger-Editor
 +
|<nowiki>- No remaining warning or errors when opening OAS YAML</nowiki>
 +
|-
 +
|Swagger-CLI
 +
| - Part of CI validation done via Jenkins in the repository
 +
- Swagger-CLI reports no error/warnings
 +
 
 +
- <code>swagger-cli validate <oas-file.yaml></code>
 +
|-
 +
|Speccy
 +
| - Part of CI validation done via Jenkins in the repository
 +
- Speccy reports no error/warnings
 +
 
 +
- <code>speccy lint "<oas-file.yaml>"</code>
 +
|}
 +
 
 +
=== Level 2 Validation - Swagger UI ===
 +
{| class="wikitable"
 +
!Validation Tool
 +
!Validation Criteria
 +
|-
 +
|Swagger-UI
 +
|<nowiki>- No error when viewing the file via Swagger UI</nowiki>
 +
 
 +
- Thorough visual inspection of endpoints and data model
 +
|}
 +
 
 +
=== Level 3 Validation - Code Generation ===
 +
{| class="wikitable"
 +
!Validation Tool
 +
!Validation Criteria
 +
|-
 +
|swagger-codegen-cli-v3
 +
|<nowiki>- From CLI tool: generate client API library/package and server stub for Java and Golang</nowiki>
 +
 
 +
- Code generation succeeds
 +
 
 +
- Extra: execute server stub without crash
 +
 
 +
- Example Golang Server & Client
 +
 
 +
<code>java -jar swagger-codegen-cli-v3.jar generate -i <oas-file.yaml> -l go-server -o ./server-gen -DpackageName=server</code>
 +
 
 +
<code>java -jar swagger-codegen-cli-v3.jar generate -i <oas-file.yaml> -l go -o ./client-gen -DpackageName=client</code>
 +
|}
 +
 
 +
== Proto3 Generation & Validation ==
 +
Some API representations include a Protobuf representation along with the OAS3 representation; this representation is generated using the tool Open API Generator
 +
 
 +
=== Proto3 Generation ===
 +
{| class="wikitable"
 +
!Generation Tool
 +
!Generation Criteria
 +
|-
 +
|openapi-generator-cli-5.0.0-beta2
 +
|<nowiki>- From CLI: generate Proto3 representation</nowiki>
 +
 
 +
- Proto3 generation succeeds
 +
 
 +
- <code>java -jar openapi-generator-cli-5.0.0-beta2.jar generate -i <oas-file.yaml> -g protobuf-schema -o proto/ --package-name <mec1234></code>
 +
 
 +
- Full inspection of the generated Proto3 is mandatory
 +
 
 +
- At the time of writing, some Proto3 generated files needs to be touched manually: enumerations, structures containing allOf, oneOf, anyOf
 +
|}
 +
 
 +
=== Proto3 Validation ===
 +
{| class="wikitable"
 +
!Validation Tool
 +
!Validation Criteria
 +
|-
 +
|protoc
 +
|<nowiki>- From CLI: generate source code from Proto3 representation</nowiki>
 +
 
 +
- Code generation succeeds
 +
 
 +
- Sometimes, when generating code, it may be necessary to include the file [https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/empty.proto empty.proto]
 +
 
 +
Example generate Go and Java
 +
<pre>
 +
mkdir go java
 +
 
 +
protoc --go_out=go --java_out=java services/*
 +
 
 +
protoc --go_out=go --java_out=java models/*
 +
</pre>
 +
|}
 +
 
 +
== Branching strategy ==
 +
Development follows a simplified version of the popular [https://nvie.com/posts/a-successful-git-branching-model/ GitFlow]
 +
* <code>master</code> - released version of the OAS representation
 +
* <code>develop</code> - final versions of OAS representations development, before they are approved to be merged  on master
 +
* <code>feature branch</code> - work branch, the final result should be merged on develop
 +
 
 +
For example, work performed by a Specialist Task Force (stf123)
 +
# STF - From <code>develop</code>, create a <code>stf123</code> <code>feature branch</code>
 +
# STF - Once work is completed & validation pass, merge the <code>stf123</code> to <code>develop</code>
 +
# STF - Create a Merge Request from <code>develop</code> to <code>master</code>
 +
# SC - A member from the STF Steering Committee will perform the final merge
 +
# SC - Apply a label corresponding the the GS version
 +
 
 +
== Folder structure and file names ==
 +
 
 +
=== Repository structure ===
 +
<pre>
 +
repository_root                    i.e. MEC
 +
├── LICENSE
 +
├── proto3                          Protobuf representation generated from OAS yaml
 +
│   ├── models                      Protobuf models
 +
│   ├── README.md                  Protobuf general info
 +
│   └── services                    Protobuf services
 +
├── README.md                      General API information
 +
├── MEC-XYZ.json                    Auto-generated JSON (From Swagger Editor -> Save As json)
 +
└── MEC-XYZ.yaml                    Definition flat file used by Swagger-UI, Swagger-Editor, Swagger-CodeGen, etc.
 +
</pre>''NOTE: Naming of of the JSON and YAML files is generally chosen by the DECODE group and does not necessarily follow the pattern outlined above.''

Latest revision as of 07:53, 23 February 2021

OpenAPI Specification (OAS) development guidelines for ETSI MEC ISG API specifications.

(More general recommendations can be found at Forge OpenAPI Guidelines)

Latest versions

Latest versions of the OAS representations are available in their respective repositories (here)

Latest CI test results of the OAS representations are available here

Contributing

If you are willing to contribute or if you want more information, contact ETSI CTI.

ETSI CTI has also made a YouTube tutorial available (based on NFV) that provides guidance on how to contribute for MEC.

API development

General guidelines

As of January 2021, OAS representations have been updated to reflect the latest released versions of their corresponding GS (e.g. v2.1.x); the representations follow OpenAPI 3.0 specification language.

The following figure provides a high level overview of the OAS creation and usage.

ToolchainOverview.png

As shown on the figure, OAS YAML file is used in 3 different contexts:

  1. To accurately reflect the GS API as released by ETSI MEC ISG
  2. To provide graphical UI documentation as an input to SwaggerUI JS Application
  3. To provide out-of-the-box API client library/package or server stub as an input to code generation tools

As such, when developing or updating the OAS YAML representations, the 3 usage described above should be considered.

The following sections provide guidelines on how to develop and validate OAS represnetaions.

OAS YAML Generation

Initial OAS generation involves using the ETSI maintained doc2oas tool; the figure below shows the steps involved in using this tool.

The following figure outlines doc2oas usage.

Doc2oasToolchain.png

Step 1 - Create MEC-XYZ_config.yaml file

  • this file contains all external types not defined in the GS in (OAS language)
  • existing config files can be used as a template, types can be re-used from existing config files
  • to quickly identify external types, Step2 can be performed with an empty config file - the resulting yaml file will show all unresolved types in the Swagger Editor

Step 2 - Use the doc2oas.sh script to generate MEC-XYZ.yaml

  • the new specification must first be added to doc2oas.sh script

Step 3 - Add endpoints as per the GS specification

  • follow the model from existing specifications
  • the end result should not show any error in Swagger Editor

OAS YAML Validation

Once the YAML file is available, the following validation steps should be performed to verify the validity in the different usage contexts

Level 1 Validation - OAS syntax

Validation Tool Validation Criteria
Swagger-Editor - No remaining warning or errors when opening OAS YAML
Swagger-CLI - Part of CI validation done via Jenkins in the repository

- Swagger-CLI reports no error/warnings

- swagger-cli validate <oas-file.yaml>

Speccy - Part of CI validation done via Jenkins in the repository

- Speccy reports no error/warnings

- speccy lint "<oas-file.yaml>"

Level 2 Validation - Swagger UI

Validation Tool Validation Criteria
Swagger-UI - No error when viewing the file via Swagger UI

- Thorough visual inspection of endpoints and data model

Level 3 Validation - Code Generation

Validation Tool Validation Criteria
swagger-codegen-cli-v3 - From CLI tool: generate client API library/package and server stub for Java and Golang

- Code generation succeeds

- Extra: execute server stub without crash

- Example Golang Server & Client

java -jar swagger-codegen-cli-v3.jar generate -i <oas-file.yaml> -l go-server -o ./server-gen -DpackageName=server

java -jar swagger-codegen-cli-v3.jar generate -i <oas-file.yaml> -l go -o ./client-gen -DpackageName=client

Proto3 Generation & Validation

Some API representations include a Protobuf representation along with the OAS3 representation; this representation is generated using the tool Open API Generator

Proto3 Generation

Generation Tool Generation Criteria
openapi-generator-cli-5.0.0-beta2 - From CLI: generate Proto3 representation

- Proto3 generation succeeds

- java -jar openapi-generator-cli-5.0.0-beta2.jar generate -i <oas-file.yaml> -g protobuf-schema -o proto/ --package-name <mec1234>

- Full inspection of the generated Proto3 is mandatory

- At the time of writing, some Proto3 generated files needs to be touched manually: enumerations, structures containing allOf, oneOf, anyOf

Proto3 Validation

Validation Tool Validation Criteria
protoc - From CLI: generate source code from Proto3 representation

- Code generation succeeds

- Sometimes, when generating code, it may be necessary to include the file empty.proto

Example generate Go and Java

mkdir go java

protoc --go_out=go --java_out=java services/*

protoc --go_out=go --java_out=java models/*

Branching strategy

Development follows a simplified version of the popular GitFlow

  • master - released version of the OAS representation
  • develop - final versions of OAS representations development, before they are approved to be merged on master
  • feature branch - work branch, the final result should be merged on develop

For example, work performed by a Specialist Task Force (stf123)

  1. STF - From develop, create a stf123 feature branch
  2. STF - Once work is completed & validation pass, merge the stf123 to develop
  3. STF - Create a Merge Request from develop to master
  4. SC - A member from the STF Steering Committee will perform the final merge
  5. SC - Apply a label corresponding the the GS version

Folder structure and file names

Repository structure

repository_root                     i.e. MEC
├── LICENSE
├── proto3                          Protobuf representation generated from OAS yaml
│   ├── models                      Protobuf models
│   ├── README.md                   Protobuf general info
│   └── services                    Protobuf services
├── README.md                       General API information
├── MEC-XYZ.json                    Auto-generated JSON (From Swagger Editor -> Save As json)
└── MEC-XYZ.yaml                    Definition flat file used by Swagger-UI, Swagger-Editor, Swagger-CodeGen, etc.

NOTE: Naming of of the JSON and YAML files is generally chosen by the DECODE group and does not necessarily follow the pattern outlined above.