Building a Bulletproof Multi-Platform Content Publishing Architecture

If you’ve ever written a high-quality technical article, you know the drill. You finish the draft, hit publish on your personal blog, and then the real work begins. You copy the text into Medium, only to find your code blocks lost their syntax highlighting. You paste it into Dev.to, and realize your cover image is broken because they require images to be uploaded to their specific CDN. You try Hashnode, and the GraphQL mutation rejects your payload because of a missing canonical URL tag.

Copy-pasting and manually tweaking content across platforms is a massive waste of time and a breeding ground for inconsistencies. As developers building SaaS products or personal brands, we need a unified engine. We need a "Write Once, Publish Everywhere" (WOPE) architecture.

In this article, we’ll dive deep into designing a robust, multi-platform content publishing system. We will cover content normalization, the adapter pattern for disparate APIs, and resilient job queuing.

The Core Architecture: Normalization, Adaptation, and Execution

A scalable publishing pipeline isn't just a script that loops through an array of API keys. It requires a decoupled architecture that separates the content representation from the platform delivery. The architecture consists of three main phases:

• Normalization Engine: Converts raw Markdown/MDX into a normalized Abstract Syntax Tree (AST), resolving platform-specific quirks (like image hosting or liquid tags).
• Adapter Layer: Translates the normalized payload into the specific API formats required by Medium (GraphQL), Dev.to (REST), and Hashnode (GraphQL).
• Execution & Scheduling Layer: A message queue (like BullMQ) that handles rate limiting, retries, idempotency, and scheduled publishing.

Let’s break down each layer with production-ready code.

Phase 1: Content Normalization via AST Transformation

The biggest mistake developers make is treating Markdown as a raw string. Markdown parsers have subtle differences. To build a reliable system, we must parse the Markdown into an AST, transform it, and serialize it back. We'll use the unified ecosystem (specifically remark) to handle this.

A common pitfall is image hosting. Medium rejects external image URLs, and Dev.to requires images to be uploaded to their CDN to prevent hotlinking. Instead of handling this in the API adapter, we handle it in the normalization phase.

Here is a robust remark plugin that extracts external image URLs, flags them for a secondary upload queue, and replaces them with platform-specific placeholders.

import { visit } from 'unist-util-visit';
import { Plugin } from 'unified';
import { Root, Image } from 'mdast';

// Define a custom node type for our upload placeholders
interface UploadPlaceholder {
  type: 'uploadPlaceholder';
  originalUrl: string;
  placeholderId: string;
}

export const remarkImageExtractor: Plugin<[], Root> = () => {
  return (tree, file) => {
    const imageQueue: { originalUrl: string; placeholderId: string }[] = [];

    visit(tree, 'image', (node: Image, index, parent) => {
      if (node.url.startsWith('http')) {
        const placeholderId = `img_${Math.random().toString(36).substring(7)}`;

        // Queue the image for async uploading to platform CDNs
        imageQueue.push({ originalUrl: node.url, placeholderId });

        // Replace the image node with a placeholder that the adapter will resolve
        const placeholder: any = {
          type: 'uploadPlaceholder',
          originalUrl: node.url,
          placeholderId,
          alt: node.alt,
          title: node.title,
        };

        if (parent && typeof index === 'number') {
          parent.children[index] = placeholder;
        }
      }
    });

    // Attach the queue to the file data for the orchestrator to process
    file.data.imageQueue = imageQueue;
  };
};

By the time the content reaches the adapter, all external images have been swapped with {{upload_image:img_xxx}} placeholders. The orchestrator processes the imageQueue, uploads the images to the target platform's API, and replaces the placeholders with the final CDN URLs.

Phase 2: The Adapter Pattern for Platform APIs

Every platform has a different API paradigm. Dev.to uses REST, Medium uses GraphQL, and Hashnode uses a slightly different GraphQL schema. Hardcoding these into a single service violates the Open/Closed Principle. We need an Adapter pattern. We define a strict contract for what a "published post" requires, and let each adapter handle the platform-specific translation.

import { z } from 'zod';

// Strict schema for our normalized internal payload
export const PublishPayloadSchema = z.object({
  title: z.string().min(1),
  contentMarkdown: z.string().min(1),
  tags: z.array(z.string()).max(5), // Dev.to limits to 5 tags
  canonicalUrl: z.string().url().optional(),
  publishedAt: z.date().optional(),
  coverImageUrl: z.string().url().optional(),
});

export type PublishPayload = z.infer<typeof PublishPayloadSchema>;

export interface PlatformAdapter {
  readonly name: string;
  publish(payload: PublishPayload): Promise<{ postId: string; url: string }>;
  update(postId: string, payload: Partial<PublishPayload>): Promise<void>;
}

export class DevToAdapter implements PlatformAdapter {
  readonly name = 'dev.to';
  private apiKey: string;

  constructor(apiKey: string) {
    this.apiKey = apiKey;
  }

  async publish(payload: PublishPayload) {
    const response = await fetch('https://dev.to/api/articles', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'api-key': this.apiKey,
      },
      body: JSON.stringify({
        article: {
          title: payload.title,
          body_markdown: payload.contentMarkdown,
          published: true,
          tags: payload.tags,
          canonical_url: payload.canonicalUrl,
          series: null, // Dev.to specific
        },
      }),
    });

    if (!response.ok) {
      const errorText = await response.text();
      throw new Error(`Dev.to API Error: ${response.status} - ${errorText}`);
    }

    const data = await response.json();
    return { postId: data.id.toString(), url: data.url };
  }

  async update(postId: string, payload: Partial<PublishPayload>) {
    // Implementation for PUT /api/articles/{id}
    // ...
    throw new Error('Not implemented');
  }
}

Notice how the DevToAdapter maps our internal PublishPayload to Dev.to's specific article wrapper. If we add Hashnode tomorrow, we just write a HashnodeAdapter that maps the same payload to their GraphQL mutation. The core business logic remains untouched.

Phase 3: Resilient Execution with BullMQ

Publishing to multiple APIs simultaneously is a recipe for rate-limit bans and partial failures. If Dev.to succeeds but Medium fails, you end up with fragmented content. Furthermore, API calls are slow. You shouldn't block the user's HTTP request while waiting for three external APIs. We must offload this to a background queue.

We'll use BullMQ (backed by Redis) to handle job scheduling, retries, and concurrency control.

import { Queue, Worker, Job } from 'bullmq';
import IORedis from 'ioredis';
import { DevToAdapter, PublishPayload } from './adapters';

const connection = new IORedis(process.env.REDIS_URL!);

// 1. Define the Queue
export const publishingQueue = new Queue('multi-platform-publish', { connection });

// 2. Define the Job Data Type
interface PublishJobData {
  payload: PublishPayload;
  platforms: string[]; // e.g., ['dev.to', 'medium']
  idempotencyKey: string; // Crucial for preventing duplicate posts
}

// 3. The Worker
export const publishingWorker = new Worker<PublishJobData>(
  'multi-platform-publish',
  async (job: Job<PublishJobData>) => {
    const { payload, platforms, idempotencyKey } = job.data;

    // Check idempotency key in DB to ensure we haven't already processed this exact draft
    const alreadyPublished = await db.checkIdempotency(idempotencyKey);
    if (alreadyPublished) {
      console.log(`Job ${job.id} already processed. Skipping.`);
      return { status: 'skipped' };
    }

    const adapters = getAdaptersForPlatforms(platforms);
    const results: Record<string, any> = {};

    // Process sequentially or with controlled concurrency to respect rate limits
    for (const adapter of adapters) {
      try {
        console.log(`Publishing to ${adapter.name} ...`);
        const result = await adapter.publish(payload);
        results[adapter.name] = result;

        // Add a small delay between API calls to avoid triggering rate limits
        await new Promise(resolve => setTimeout(resolve, 1500));
      } catch (error) {
        console.error(`Failed to publish to ${adapter.name}:`, error);
        // Depending on your business logic, you might throw here to trigger a BullMQ retry,
        // or log and continue to publish to the remaining platforms.
        // For critical platforms, throw an error to trigger exponential backoff.
        if (adapter.name === 'dev.to') {
          throw error;
        }
      }
    }

    // Mark idempotency key as processed
    await db.markIdempotencyProcessed(idempotencyKey);

    return { status: 'success', results };
  },
  {
    connection,
    limiter: {
      max: 10, // Max 10 jobs
      duration: 60000, // per 60 seconds (Global rate limiting)
    },
  }
);

This worker implementation includes three critical features for production systems:

• Idempotency Keys: Prevents duplicate posts if a job is retried or manually re-queued.
• Global Rate Limiting: The limiter config ensures we don't blast the platforms with concurrent requests.
• Inter-job Delays: The setTimeout between adapter calls prevents hitting per-minute rate limits on specific endpoints.

Common Pitfalls and Performance Considerations

When building this architecture, you will inevitably run into a few edge cases. Here is how to avoid them:

• The Canonical URL Trap: Always set the canonical_url to your own domain. If you publish to Medium and Dev.to without a canonical URL, those platforms will index the content first, destroying your personal blog's SEO.
• Tag Normalization: Dev.to limits tags to 5 and requires them to be lowercase without spaces. Medium uses "topics". You need a normalization function in your payload validation (using Zod, as shown above) to strip spaces and enforce limits before the data reaches the adapters.
• Handling API Token Expiry: OAuth tokens for Medium expire. Your adapter layer needs a mechanism to refresh tokens or alert the user via a webhook when a token becomes invalid, rather than just failing silently in the queue.
• Performance: Image processing is CPU intensive. If you are resizing or optimizing images before