― Edward Snowden
Imagine a distributed surveillance system where recorded video files are uploaded to a central server at regular intervalls.
Due to limitations of the transport protocol, video files must be split up in chunks and no chunk may exceed 1 GB (10^9 bytes). On top of that, in high-load scenarios, the server might shorten a chunk even more, in which case instead of N bytes only K bytes are transmitted. Naturally, the N-K bytes that were not transmitted need to be sent with the next chunk upload.
Everything works fine, all unit and system tests passed. Once deployed, however, sysadmins from the central server team started lamenting that the video files were arriving at a glacial pace. What’s wrong with this code?
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 |
// Establishes a connection to a video upload server. // Returns a handle to the server connection or -1 if no connection // could be established. int connectToServer(const char* url, int port); // Uploads at most 'length' bytes of 'data' to an upload server connection. // Returns the actual number of bytes uploaded (<= length) or a value <= 0 if // the connection is broken. int uploadToServer(int serverHandle, const char* data, int length); // Waits for and retrieves the next stored video. // Returns a pointer to a contiguous video blob or NULL if there is temporarily // no video. The size of the video blob in bytes is stored in the 'length' out // parameter. const char* getNextVideo(int* length); static const char* const serverUrl = "http://video.upload.server.com"; static const int serverPort = 1234; static const int maxChunkSize = 10^9; ... // Open connection to server. int serverHandle = connectToServer(serverUrl, serverPort); if (serverHandle >= 0) { bool continueToUpload = true; while (continueToUpload) { // Get next video. int remainingVideoLength = 0; const char* p = getNextVideo(&remainingVideoLength); // If there is a video. if (p != NULL) { // Upload video in chunks. while (remainingVideoLength > 0) { int bytesToUpload = remainingVideoLength; if (bytesToUpload > maxChunkSize) { bytesToUpload = maxChunkSize; } int uploadedBytes = uploadToServer(serverHandle, p, bytesToUpload); if (uploadedBytes > 0) { p += uploadedBytes; remainingVideoLength -= uploadedBytes; } else { fprintf(stderr, "Uploading failed, aborting"); continueToUpload = false; break; } } } } } else { fprintf(stderr, "Failed to connect to %s:%d", serverUrl, serverPort); } |