Skip to content

README.adoc

Latest commit

 

History

History
84 lines (51 loc) · 3.54 KB

README.adoc

File metadata and controls

84 lines (51 loc) · 3.54 KB
Raw

Camel Spring Boot Example: HTTP Large File Streaming

Introduction

This project contains logic to handle large HTTP data streams in download, upload, and proxy scenarios. The aim is to show Camel’s ability to process streams without exhausting JVM memory.

Use cases

Note

Streaming mode opens up many interesting use cases, for example, data transformation on very large data structures, applying selective/discarding rules, split/merging of data, multiplexing data streams, and others.

This example only covers the basics of enabling streaming mode. The two implemented scenarios using Camel in streaming mode are:

  • HTTP data downloads

  • HTTP data uploads

Camel may act as the end system responsible to locally/remotely store the data stream. Or it may also act as a proxy system, passing the responsibility downstream.

The critical data handling happens in the numbered 1) and 2) positions illustrated below.

Download scenario

uc download

Upload scenario

uc upload

How to run

To demonstrate Camel can handle larger data streams than memory allocated to the JVM, we need to start it with low memory settings.

To run it follow the commands below:

Follow these steps to run the example:

  1. Decide which scenario you want to run: download or upload.

  2. Navigate to the corresponding folder and run both the backend and proxy servers with low memory settings (make sure ports 8080 and 9000 are available):

    mvn spring-boot:run -Dspring-boot.run.jvmArguments="-Xms200m -Xmx200m"

  3. From the client directory, send a request using a large data stream. (See detailed instructions in client/Readme.adoc.)

  4. Stop Camel Spring Boot and inspect the result file.

    Camel Spring Boot should process the HTTP byte stream and dump it in a file in the 'client' directory.

A note on jailStartingDirectory

The upload backend writes the received stream with to("file:../client?fileName=output&jailStartingDirectory=false").

jailStartingDirectory is Camel’s built-in path-traversal guard for file: endpoints: enabled by default, it refuses to read/write any resolved path that falls outside the endpoint’s configured starting directory, which is what stops a crafted CamelFileName header from writing (or reading) files outside that directory via ../ sequences.

It is disabled here only because the starting directory itself (../client) is expressed as a ..-relative path, which Camel’s containment check now unconditionally rejects regardless of destination — there was no way to keep the check enabled without also changing how the target directory is expressed. That’s safe in this example specifically because the written file name (output) is a hardcoded constant, never derived from exchange or header data, so there is nothing here for the check to actually protect against.

Important

Do not copy jailStartingDirectory=false into routes where the file name (or the directory) is derived from exchange data, headers, or any external input. Doing so reopens the exact path-traversal risk the option exists to prevent. Keep the default (true) in that case, and resolve the target directory to an absolute/canonical path instead of a ..-relative one, so the containment check stays active and meaningful.

Help and contributions

If you hit any problem using Camel or have some feedback, then please let us know.

We also love contributors, so get involved :-)

The Camel riders!