How To Handle File Uploading

How to implement the file upload process in a REST API

Reading Time: 14 minutes

This mail was written by one of the stars in our developer community, Thiago Santana.

File-sharing is one of the most uncomplicated ways to perform system integration. In the context of spider web applications, we telephone call "upload" the procedure in which a user sends data/files from a local computer to a remote computer.

Sometimes we need to betrayal, in our Balance API, an upload operation that allows the manual of:

Binary files (of any kind)
The meta information related to it (e.g., proper name, content type, size and then on…)
Perhaps some additional information for performing business logic processing

But we would similar all this information to arrive on the server in the same request. This sounds similar a challenge, doesn't it?

This guide volition demonstrate i strategy, among many, to implement this integration scenario using resources provided by Mule components.

Soap Legacy – Identifying Reusable Concepts

Just to contextualize, this strategy was inspired past the architecture of a legacy project that I needed to maintain. The project needed to transmit binary files in SOAP Web Services, without using MTOM.

When we do non use MTOM with Lather, and the files are transmitted as a MIME attachment of the payload (similar to the process of sending emails containing attachments [meet fig.1]), the implementation of the protocol converts the contents of the file that needs to be transmitted, and the result of this conversion is also a binary string in the Base64 format.

After the conversion, the Cord is deposited inside an XML tag, which is part of the Body content of the payload that is being transmitted to the remote server [see fig2].

There is an reward to using this strategy, which is the ability to transmit this tag in the same payload that may contain other data, such as the data needed to process business organization rules, or better withal contain the meta information of the file.

Since all the necessary information is in the same payload, the process of reading and/or parsing the message is simplified. All of this is in a single request handled past the service, which is very important for the processing economy and helps avoid maintaining and/or managing transactions handled by the service, in other words, it conforms to the Stateless beliefs.

This is very desirable because information technology conforms to one of the nifty features of the HTTP protocol (which was designed to be Stateless) and allows the transaction to be completely processed in one request. As a event, it will not be necessary to asking additional information for the service consumer; in add-on, nosotros tin all the same do function of the treatment procedure internally, asynchronously, without prolonging the client response time or timeout.

Finally, this strategy likewise provides server resource savings since there is no demand to request additional information regarding the transaction in progress.

I thought that some of the advantages identified in the architecture applied in the legacy solution context, and and so I congenital a style to port this architecture to a REST API, applying also significant improvements.

Implementation – Porting the Solution to REST API

When defining any type of API, nosotros should consider that, when making utilise of whatsoever interface, consumers usually expect to find availability, simplicity, and stability.Defining a RAML contract is ane of the means to plant guidelines that favor the structure of a REST API which offers simplicity and stability from the start.

Similar to WSDL southward, when we define and make available a RAML, we give our consumers a better chance to set for it, as well equally place difficulties and provide suggestions for improvements for future versions of the contract. Using gear up-fabricated infrastructures, such equally the Mulesoft's Anypoint API Designer, the consumer tin can also examination this RAML by generating an endpoint with mock information (provided in the RAML itself), testing the consuming office of the interface. You lot are able to do all of this before yous make a minimal implementation available to run on some server.

To implement the in a higher place-proposed scenario, we define the RAML contract containing the Mail operation for the resource "file," something like:

For a resources like this, we can accept the following HTTP asking example:

post payload

Below, I explain the flows that brand upwards the Mule project of the REST API, which is running as Mule 3.eight.one EE runtime:

Main Flow:

It consists of an HTTP entering endpoint configured to handle requests on port 8081, an APIKit Router, and a Reference Exception Strategy.

main_flow 5

Postal service Flow:

The APIKit Router sends HTTP Post requests to this stream. It consists of two triggers (Menstruation References) for the buildFileData stream and filesSplitter, DataWeave which prepares the response that is returned to the consumer, and a Logger to register on the console that response.

post_flow 6

Build FilesData Menstruation:

Declaration of the FilesContent variables (a Java HashMap) and FilesData (a simple String var). A ForEach traverses the inboundAttachment of the current message (a multipart-formData) to split the FileContent [] from the filesData. At that place is besides a component to remove the rootMessage variable generated during the interaction of the ForEach.

Lastly, DataWeave performs the merge of the original FilesData Payload with the FilesContent data. This is done here to simplify for the consumer the activity of passing binary data inside a JSON. DataWeave also disregards any by FileContent [] that is not declared as used in the "fileContentIndex" field in the received FilesData JSON. This helps filter information and avoids the apply of invalid data.

buildFilesData_flow

Files Splitter Menstruation:

This converts the received JSON FilesData to a congenital-in Mule Coffee object. This object is partitioned, each office represents a file that will be sent to the outbound VM component (where each file is processed separately). The Drove Aggregator component is responsible for gathering the enriched payloads in the same structure.

filesSplitter_flow

Upload File Flow:

As suggested by the MuleSoft documentation, we can use an inbound VM to handle requests that originate from a Bulletin Splitter. Inside this Menses, each file is pre-processed and and so sent to Amazon S3 by the Create Object operation. The pre-process consists of converting the Base64 binary String from the "fileContent" field of the payload into a BiteArray. The outcome of this conversion, which, in this case, isperformed by the base64-decoder component, is stored in a variable.

This variable satisfies the Create Object component, along with the file and saucepan names. Because this operation doesn't return a usable response, we have to add a call to Become Object component that allows us to recollect data from the file that was stored in the storage.

We can complete this step with the assist of DataWeave–– the data received at the beginning of this flow is enriched with the FileSize and HttpURI data that was received in the response of the S3 Get Object operation.

uploadFile_flow

Exception Mapping Flow:

Mule automatically generates this flow when nosotros create a project, providing some RAML file to the APIKit. It offers some exception treatments handled automatically by APIKit.

exceptionMapping_flow

Testing the Solution

Near unproblematic Balance API tests, especially those involving Go operations, tin easily be done by command line utilities; for example, curl. Stress tests tin can be generated with the assistance of the JMeter tool. Tests tin also exist performed on newer versions of SoapUI.

Delight note that I opted to perform the tests with the help of the Postman tool (an extension for the Google Chrome browser).

You can verify that we can submit more than i fileContent[] as grade param, but just the indexes reported equally being used in the JSON passed in the form param "filesData" will be candy by API––all others will be ignored. If no exception happens, we receive a JSON containing the processed data and HTTP status 201.

file upload