Difference between revisions of "OpenAPI development guidelines"

From MECwiki
Jump to: navigation, search
Line 1: Line 1:
OpenAPI development guidelines for ETSI MEC ISG API specifications.
+
OpenAPI Specification (OAS) development guidelines for ETSI MEC ISG API specifications.
More general recommendations can be found at [https://forge.etsi.org/wiki/index.php/Main_Page Forge OpenAPI Guidelines].
+
 
 +
''(More general recommendations can be found at [https://forge.etsi.org/wiki/index.php/Main_Page Forge OpenAPI Guidelines])''
  
 
= Latest versions =
 
= Latest versions =
Line 10: Line 11:
 
= Contributing =
 
= Contributing =
  
If you are willing to contribute or if you want more information, contact [mailto:cti_support@etsi.org?subject=MEC%20openapis%20development: ETSI CTI]. ETSI CTI have also made a [https://www.youtube.com/watch?v=YXO0R61Dcg8 YouTube] tutorial available, which although based on NFV rather than MEC provides a valuable overall guide.
+
If you are willing to contribute or if you want more information, contact [mailto:cti_support@etsi.org?subject=MEC%20openapis%20development: ETSI CTI].  
 +
 
 +
ETSI CTI has also made a [https://www.youtube.com/watch?v=YXO0R61Dcg8 YouTube] tutorial available (based on NFV) that provides guidance on how to contribute for MEC.
  
 
= API development =
 
= API development =
Line 16: Line 19:
 
== General guidelines ==
 
== General guidelines ==
  
MEC APIs are currently described using the [https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md OpenAPI 2.0 specification]. Members of the ETSI MEC ISG can view a tutorial [https://docbox.etsi.org/ISG/MEC/05-CONTRIBUTIONS/2017//MEC(17)000541_MEC023_Creating_a_OpenAPI_spec_compliant_API_description.pptx presentation] on creating a OpenAPI based description using the MEC-011 Mp1 API as the basis. That can be made available outside the ISG on request.
+
As of January 2021, OAS representations have been updated to reflect the latest released versions of their corresponding GS (e.g. v2.1.x);  the representations follow [https://swagger.io/specification/ OpenAPI 3.0 specification] language.  
  
A migration to the latest version of the [https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md OpenAPI 3.0 specification] is being actively pursued within the ISG. However, it is desirable that auto-generation of client and server stubs are supported through [https://github.com/swagger-api/swagger-codegen swagger-codegen], which is still in development (reference 3.0.0_enhancements branch).
+
The following figure provides a high level overview of the OAS creation and usage.
  
=== Root API files ===
+
<INSERT - ToolchainOverview.png>
  
Since the specification requires the <code>apiVersion</code> be after <code>apiName</code> in the resource path, each API will have a separate OpenAPI root file.
+
As shown on the figure, OAS YAML file is used in 3 different contexts:
 +
# To accurately reflect the GS API as released by ETSI MEC ISG
 +
# To provide graphical UI documentation as an input to SwaggerUI JS Application
 +
# To provide out-of-the-box API client library/package or server stub as an input to code generation tools
  
In detail, paths are required to start with:
+
As such, when developing or updating the OAS YAML representations, the 3 usage described above should be considered.
 +
 
 +
The following sections provide guidelines on how to develop and validate OAS represnetaions.
 +
 
 +
== OAS YAML Generation ==
 +
Initial OAS generation involves using the ETSI maintained doc2oas tool; the figure below shows the steps involved in using this tool.
 +
 
 +
The following figure outlines doc2oas usage.
 +
 
 +
<INSERT - doc2oasToolchain.png>
 +
 
 +
Step 1 - Create <code>MEC-XYZ_config.yaml</code> file -- this file contains all external types not defined in the GS in (OAS language)
 +
* existing config files can be used as a template, many types can be re-used
 +
* to quickly identify external types, Step2 can be performed with an empty config file - the resulting yaml file will show all unresolved types in the OAS editor
 +
 
 +
Step 2 - Use the <code>doc2oas.sh</code> script to generate <code>MEC-XYZ.yaml</code>
 +
* the new specification must first be added to <code>doc2oas.sh</code> script
 +
 
 +
Step 3 - Add endpoints as per the GS specification
 +
* follow the model from existing specifications
 +
* the end result should not show any error in Swagger Editor
 +
 
 +
== OAS YAML Validation ==
 +
Once the YAML file is available, the following validation steps should be performed to verify the validity in the different usage contexts
 +
 
 +
=== Level 1 Validation - OAS syntax ===
 +
{| class="wikitable"
 +
|+
 +
!Validation Tool
 +
!Validation Criteria
 +
|-
 +
|Swagger-Editor
 +
|<nowiki>- No remaining warning or errors when opening OAS YAML</nowiki>
 +
|-
 +
|Swagger-UI
 +
|<nowiki>- No error when viewing the file via Swagger UI</nowiki>
 +
 
 +
- Thorough visual inspection of endpoints and data model
 +
|-
 +
|Speccy
 +
| - Part of CI validation done via Jenkins in the repository
 +
- Speccy reports no error/warnings
 +
 
 +
- <code>speccy lint "<oas-file.yaml>"</code>
 +
|}
 +
 
 +
=== Level 2 Validation - Swagger UI ===
 +
{| class="wikitable"
 +
!Validation Tool
 +
!Validation Criteria
 +
|-
 +
|Swagger-UI
 +
|<nowiki>- No error when viewing the file via Swagger UI</nowiki>
 +
 
 +
- Thorough visual inspection of endpoints and data model
 +
|}
 +
 
 +
=== Level 3 Validation - Code Generation ===
 +
{| class="wikitable"
 +
!Validation Tool
 +
!Validation Criteria
 +
|-
 +
|swagger-codegen-cli-v3
 +
|<nowiki>- From CLI tool: generate client API library/package and server stub for Java and Golang</nowiki>
 +
 
 +
- Code generation succeeds
 +
 
 +
- Extra: execute server stub without crash
 +
 
 +
- Example Golang Server & Client
 +
 
 +
<code>java -jar swagger-codegen-cli-v3.jar generate -i <oas-file.yaml> -l go-server -o ./server-gen -DpackageName=server</code>
 +
 
 +
<code>java -jar swagger-codegen-cli-v3.jar generate -i <oas-file.yaml> -l go -o ./client-gen -DpackageName=client</code>
 +
|}
 +
 
 +
== Proto3 Generation & Validation ==
 +
Some API representations include a Protobuf representation along with the OAS3 representation; this representation is generated using the tool Open API Generator
 +
 
 +
=== Proto3 Generation ===
 +
{| class="wikitable"
 +
!Generation Tool
 +
!Generation Criteria
 +
|-
 +
|openapi-generator-cli-5.0.0-beta2
 +
|<nowiki>- From CLI: generate Proto3 representation</nowiki>
 +
 
 +
- Proto3 generation succeeds
 +
 
 +
- <code>java -jar openapi-generator-cli-5.0.0-beta2.jar generate -i <oas-file.yaml> -g protobuf-schema -o proto/ --package-name <mec1234></code>
 +
 
 +
- Full inspection of the generated Proto3 is necessary
 +
 
 +
- At the time of writing, some Proto3 generated files needs to be touched manually: enumerations, structures containing allOf, oneOf, anyOf
 +
|}
 +
 
 +
=== Proto3 Validation ===
 +
{| class="wikitable"
 +
!Validation Tool
 +
!Validation Criteria
 +
|-
 +
|protoc
 +
|<nowiki>- From CLI: generate source code from Proto3 representation</nowiki>
 +
 
 +
- Code generation succeeds
 +
 
 +
- Example generate Go and Java
 
<pre>
 
<pre>
<apiRoot>/<apiName>/<apiVersion>
+
mkdir go java
</pre>
 
  
the only way to model this with OAS is using separate files for each interface. This will also help readability and clarity.
+
protoc --go_out=go --java_out=java services/*
  
=== Definitions ===
+
protoc --go_out=go --java_out=java models/*
Definitions are currently not shared across API descriptions and there is no cross-referencing. This is also the case in the group specifications themselves.
+
</pre>
 
+
|}
=== Resources ===
 
It is encouraged to create a separate file for each resource (as is the case for Mp1 and Location), although only the overall description file is provided for most APIs.
 
  
 
== Branching strategy ==
 
== Branching strategy ==
 +
Development follows a simplified version of the popular [https://nvie.com/posts/a-successful-git-branching-model/ GitFlow]
 +
* <code>master</code> - released version of the OAS representation
 +
* <code>develop</code> - final versions of OAS representations development, before they are approved to be merged  on master
 +
* <code>feature branch</code> - work branch, the final result should be merged on develop
  
* Within the repository there will be a branch for each ''active'' version of the interfaces. A version is ''active'' if it is still under maintenance in the WG.
+
For example, work performed by a Specialist Task Force (stf123)
* Git tags will be used to label a commit on a branch with a specific revision of the deliberable.
+
# STF - From <code>develop</code>, create a <code>stf123</code> <code>feature branch</code>
 +
# STF - Once work is completed & validation pass, merge the <code>stf123</code> to <code>develop</code>
 +
# STF - Create a Merge Request from <code>develop</code> to <code>master</code>
 +
# SC - A member from the STF Steering Committee will perform the final merge
 +
# SC - Apply a label corresponding the the GS version
  
 
== Folder structure and file names ==
 
== Folder structure and file names ==

Revision as of 22:17, 19 February 2021

OpenAPI Specification (OAS) development guidelines for ETSI MEC ISG API specifications.

(More general recommendations can be found at Forge OpenAPI Guidelines)

Latest versions

Latest versions of the OAS representations are available in their respective repositories (here)

Latest CI test results of the OAS representations are available here

Contributing

If you are willing to contribute or if you want more information, contact ETSI CTI.

ETSI CTI has also made a YouTube tutorial available (based on NFV) that provides guidance on how to contribute for MEC.

API development

General guidelines

As of January 2021, OAS representations have been updated to reflect the latest released versions of their corresponding GS (e.g. v2.1.x); the representations follow OpenAPI 3.0 specification language.

The following figure provides a high level overview of the OAS creation and usage.

<INSERT - ToolchainOverview.png>

As shown on the figure, OAS YAML file is used in 3 different contexts:

  1. To accurately reflect the GS API as released by ETSI MEC ISG
  2. To provide graphical UI documentation as an input to SwaggerUI JS Application
  3. To provide out-of-the-box API client library/package or server stub as an input to code generation tools

As such, when developing or updating the OAS YAML representations, the 3 usage described above should be considered.

The following sections provide guidelines on how to develop and validate OAS represnetaions.

OAS YAML Generation

Initial OAS generation involves using the ETSI maintained doc2oas tool; the figure below shows the steps involved in using this tool.

The following figure outlines doc2oas usage.

<INSERT - doc2oasToolchain.png>

Step 1 - Create MEC-XYZ_config.yaml file -- this file contains all external types not defined in the GS in (OAS language)

  • existing config files can be used as a template, many types can be re-used
  • to quickly identify external types, Step2 can be performed with an empty config file - the resulting yaml file will show all unresolved types in the OAS editor

Step 2 - Use the doc2oas.sh script to generate MEC-XYZ.yaml

  • the new specification must first be added to doc2oas.sh script

Step 3 - Add endpoints as per the GS specification

  • follow the model from existing specifications
  • the end result should not show any error in Swagger Editor

OAS YAML Validation

Once the YAML file is available, the following validation steps should be performed to verify the validity in the different usage contexts

Level 1 Validation - OAS syntax

Validation Tool Validation Criteria
Swagger-Editor - No remaining warning or errors when opening OAS YAML
Swagger-UI - No error when viewing the file via Swagger UI

- Thorough visual inspection of endpoints and data model

Speccy - Part of CI validation done via Jenkins in the repository

- Speccy reports no error/warnings

- speccy lint "<oas-file.yaml>"

Level 2 Validation - Swagger UI

Validation Tool Validation Criteria
Swagger-UI - No error when viewing the file via Swagger UI

- Thorough visual inspection of endpoints and data model

Level 3 Validation - Code Generation

Validation Tool Validation Criteria
swagger-codegen-cli-v3 - From CLI tool: generate client API library/package and server stub for Java and Golang

- Code generation succeeds

- Extra: execute server stub without crash

- Example Golang Server & Client

java -jar swagger-codegen-cli-v3.jar generate -i <oas-file.yaml> -l go-server -o ./server-gen -DpackageName=server

java -jar swagger-codegen-cli-v3.jar generate -i <oas-file.yaml> -l go -o ./client-gen -DpackageName=client

Proto3 Generation & Validation

Some API representations include a Protobuf representation along with the OAS3 representation; this representation is generated using the tool Open API Generator

Proto3 Generation

Generation Tool Generation Criteria
openapi-generator-cli-5.0.0-beta2 - From CLI: generate Proto3 representation

- Proto3 generation succeeds

- java -jar openapi-generator-cli-5.0.0-beta2.jar generate -i <oas-file.yaml> -g protobuf-schema -o proto/ --package-name <mec1234>

- Full inspection of the generated Proto3 is necessary

- At the time of writing, some Proto3 generated files needs to be touched manually: enumerations, structures containing allOf, oneOf, anyOf

Proto3 Validation

Validation Tool Validation Criteria
protoc - From CLI: generate source code from Proto3 representation

- Code generation succeeds

- Example generate Go and Java

mkdir go java

protoc --go_out=go --java_out=java services/*

protoc --go_out=go --java_out=java models/*

Branching strategy

Development follows a simplified version of the popular GitFlow

  • master - released version of the OAS representation
  • develop - final versions of OAS representations development, before they are approved to be merged on master
  • feature branch - work branch, the final result should be merged on develop

For example, work performed by a Specialist Task Force (stf123)

  1. STF - From develop, create a stf123 feature branch
  2. STF - Once work is completed & validation pass, merge the stf123 to develop
  3. STF - Create a Merge Request from develop to master
  4. SC - A member from the STF Steering Committee will perform the final merge
  5. SC - Apply a label corresponding the the GS version

Folder structure and file names

Repository structure

repository_root                     i.e. MEC
|- Group Specification (GS) x       e.g. MEC.GS-011
   |- Readme.md                     General API information
   |- APIx.json                     Auto-generated JSON
   |- APIx.yaml                     Main OAS file (auto-generated using the APIx.split.yaml file: multi-file-swagger Mp1.split.yaml > Mp1.json), e.g. Mp1.yaml
   |- APIx.split.yaml               Definition file split into its component parts, to bring it together use [https://github.com/mohsen1/multi-file-swagger-example multi-file-swagger]
      |- definitions
         | - DataType1.yaml         e.g. CurrentTime.yaml
         | - DataType2.yaml
      |- examples                   Example responses
         | - Example1.yaml          e.g. ServiceInfo.yaml
         | - Example2.yaml
      |- externalDocs               Links to external docs, i.e. the GS
         | - index.yaml
      |- info                       General info regarding API, e.g. Copyright
         | - index.yaml
      |- parameters                 API Body/Query/Path Parameters
         | - Parameter1.yaml        e.g. Body.DnsRule.yaml
         | - Parameter2.yaml        e.g. Path.DnsRuleId.yaml
      |- paths                      Paths associated with the API
         | - PathA.yaml             High level path, e.g. Services.yaml
         | - PathA.{method}.yaml    Lower level path, e.g. Services.GET.yaml
|- Group Specification y
   |- ...
|- Group Specification z
   |- ...
|- Readme.md
|- LICENSE

File names

The naming schema provided in the examples for the "Repository structure" above is followed.