#Motivation: Thinking of Content as Data

#DIY Example with Next.js + Markdown

tsx
import fs from "fs";
import glob from "glob";
import path from "path";
import rehypeSanitize from "rehype-sanitize";
import rehypeStringify from "rehype-stringify";
import matter from "gray-matter";
 
import remarkParse from "remark-parse";
import remarkRehype from "remark-rehype";
import { unified } from "unified";
 
import type { Page, PageFrontmatter } from "../types/Page";
 
const contentDir = path.join(process.cwd(), "../content");
 
/**
 * Full paths to all content source files.
 *
 * @returns an array of all content source files
 */
function allPageFilePaths(): string[] {
	return glob.sync(path.join(contentDir, "**/*.md"));
}
 
/**
 * Given the full path to a content source file, provide a relative path from
 * the content source directory, without the file extension.
 *
 * @param filePath Full path to the content source file
 * @returns Path that can be used as a urlPath for this piece of content
 */
function buildPageUrlPath(filePath: string): string {
	const relFilePath = filePath.replace(contentDir, "");
	return relFilePath.replace(/.md$/, "");
}
 
/**
 * Read from a local file and convert it into a Page object.
 *
 * @param filePath Full path to the content source file
 * @returns processed Page object
 */
async function processPage(filePath: string): Promise<Page> {
	const rawContent = fs.readFileSync(filePath).toString();
 
	const { data, content } = matter(rawContent);
	const frontmatter = data as PageFrontmatter;
 
	const body = await unified()
		.use(remarkParse)
		.use(remarkRehype)
		.use(rehypeSanitize)
		.use(rehypeStringify)
		.process(content);
 
	const urlPath = buildPageUrlPath(filePath);
 
	return {
		...frontmatter,
		urlPath,
		body: {
			raw: content,
			html: String(body),
		},
	};
}
 
/**
 * Process and rerturn all pages.
 *
 * @returns A list of all Page objects
 */
export async function allPages(): Promise<Page[]> {
	let pages = [];
	for (const filePath of allPageFilePaths()) {
		const page = await processPage(filePath);
		pages.push(page);
	}
	return pages;
}
 
/**
 * Get an array of all possible page paths, to be used with getStaticPaths().
 *
 * @returns array of all possible page paths
 */
export async function allPagePaths(): Promise<string[]> {
	return allPageFilePaths().map((filePath) => buildPageUrlPath(filePath));
}
 
/**
 * Given a current urlPath, process the appropriate content source file as a
 * page.
 *
 * @param urlPath path to the current path
 * @returns Page object
 */
export async function pageByUrlPath(urlPath: string): Promise<Page> {
	// Build an object when the keys are paths relative to the content source and
	// the values are the original full file paths.
	const pagePathMap = Object.fromEntries(
		allPageFilePaths().map((filePath) => [buildPageUrlPath(filePath), filePath])
	);
	const page = await processPage(pagePathMap[urlPath]);
	return page;
}
 

• Write markdown processing logic, including working with content-processing libraries and their plugins. This is a lot of base-level logic and adds a number of dependencies to your project.
• Implement content validation to make sure your content matches the structure expected by your application.
• Cache processed content data to avoid unnecessary work. But caching is hard and Not caching degrades the developer experience and build performance.
• Incrementally regenerate only the content you changed to optimize rebuild/reload performance.
• Account for rich and complex content relations, so that objects can live as individual content files but be embedded as necessary when used as data.
• Most frameworks (e.g Next.js) don't support content live-reload. Building this yourself is usually very complicated.
• If you want to work with TypeScript and have strongly-typed data, you must write those types manually and hope that the content data matches your types.

#Introducing Contentlayer

#How Contentlayer Works

1. First you need to tell Contentlayer where your content lives and which format and structure it has. There are different kinds of content sources - here we're using local Markdown files.
   contentlayer.config.ts

   1

   2

   3

   4

   5

   6

   7

   8

   9

   10

   11

   12

   13

   14

   15

   16

   17

   18

   19

   tsx
   import { defineDocumentType, makeSource } from 'contentlayer/source-files'
    
   const Post = defineDocumentType(() => ({
     name: 'Post',
     filePathPattern: `**/*.md`,
     fields: {
       title: { type: 'string', required: true },
       date: { type: 'date', required: true }
     },
     computedFields: {
       url: { type: 'string', resolve: (post) => `/posts/${post._raw.flattenedPath}` },
     },
   }))
    
   export default makeSource({
     contentDirPath: 'posts',
     /*              ^^^^^^^ Directory with the Markdown files. */
     documentTypes: [Post]
   })

2. Run Contentlayer to transform the content into data. This can be done independently, or by hooking into framework server/build processes.
   posts/

   • ├── change-me.md
   • ├── click-me.md
   • └── what-is-contentlayer.md

   .contentlayer/generated/

   • ├── Post/

     • │ ├── change-me.md.json
     • │ ├── click-me.md.json
     • │ └── what-is-contentlayer.md.json
   • ├── index.d.ts
   • └── index.mjs
3. Use import to work with your content, just as you would with any other data or code in your application.
   app/posts/page.tsxapp/posts/[slug]/page.tsx

   1

   2

   3

   4

   5

   6

   7

   8

   9

   10

   11

   12

   13

   14

   15

   16

   tsx
   import { allPosts } from 'contentlayer/generated'
    
   export default function Home() {
     return (
       <div>
         <h1>All posts</h1>
         <ul>
           {allPosts.map((post) => (
             <li key={post.url}>
               <a href={post.url}>{post.title}</a>
             </li>
           ))}
         </ul>
       </div>
     )
   }

#Playground: Try it Yourself

• The content is generated into a .contentlayer/generated directory that includes JSON content caches, exported collections of data, and automatically-generated types.
• The pages/posts/[[...slug]].tsx page brings in the content directly, manipulates it as needed, and passes it onto the page component.

#Benefits of Contentlayer

• Content just works: Gone are the days of writing low-level content-processing logic. Instead, you set the rules and the content just works.
• TypeScript Support: Auto-generated TypeScript type definitions every document type.
• Content validation: Content is validated automatically with helpful error messages.
• Caching: Content builds are cached and incrementally regenerated - doing only the minimal amount of work necessary when changing your content.
• Live reload: The browser will automatically reload after changing content for those frameworks that support live reloading.
• Flexible content modeling: Contentlayer allows you to structure your content in a flexible and powerful way.

#Build Performance Comparison: Contentlayer vs DIY and Gatsby

Smaller is better/faster. Used machine: GitHub Actions

#The Future of Content(layer)

• Add support for more content sources (e.g. Contentful, Notion, ...) and even supporting multiple content sources in a single project
• Providing seamless integrations for more site frameworks (such as Vite, Remix, Astro, ...)
• Validate and evolve the core abstractions of Contentlayer to avoid breaking changes after the 1.0 release

Supported Frameworks

Supported Content Sources

#Join the Contentlayer Community

• Following the tutorial and using Contentlayer into your project.
• Looking through examples and other open-source projects created with Contentlayer.
• Joining our welcoming Discord community.
• Reporting issues in the GitHub repository - Please also consider starring the repo 🌟

#Thank You 💜

• Everyone already using Contentlayer for providing early feedback shaping the project
• All contributors for reporting and investigating issues
• Stackbit for working together to bring this project to life
• Sean Davis and Lukas Wiesehan for helping to launch Contentlayer's new website and documentation

Johannes Schickling

@schickling

Related Posts

Why Working with Content is Hard for Developers

April 18, 2022

Sean C Davis

How we work with content on the web has shifted dramatically since the early days. Yet, working with content is still too difficult for developers.

Read more