# HG changeset patch # User Guido Berhoerster # Date 1541691034 -3600 # Node ID 5d7c13e998e9d6e025b27ceed02451f715fdb6fe # Parent 341a0f4b7ce09ed05a94f9c3176f20b56193663c Create feed previews using a stream filter Instead of replacing the feed document with an XHTML preview from a content script after it has already been rendered, create an XHTML preview using a stream filter before it is passed into the rendering engine and use an XSL style sheet to convert it to HTML. This has two advantages, firstly it results in an HTMLDocument with the full HTML DOM available and secondly it avoids rendering the document twice. Refactor the feed preview creation and split parsing and rendering into seperate modules. diff -r 341a0f4b7ce0 -r 5d7c13e998e9 Makefile --- a/Makefile Sun Nov 04 10:03:05 2018 +0100 +++ b/Makefile Thu Nov 08 16:30:34 2018 +0100 @@ -36,11 +36,13 @@ NEWS \ README \ $(wildcard _locales/*/messages.json) \ - background.js \ + background.html \ content_scripts/feed-probe.js \ - content_scripts/feed-preview.js \ icons/feed-preview.svg \ $(BITMAP_ICONS) \ + js/background.js \ + js/feed-parser.js \ + js/feed-preview.js \ popup/feed-selection.js \ popup/feed-selection.html \ web_resources/style/feed-preview.css \ @@ -48,6 +50,7 @@ web_resources/style/common.css \ web_resources/style/entry-content.css \ web_resources/feed-preview.xhtml \ + web_resources/xhtml-to-html.xsl \ web_resources/images/arrow.svg .DEFAULT_TARGET = all diff -r 341a0f4b7ce0 -r 5d7c13e998e9 background.html --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/background.html Thu Nov 08 16:30:34 2018 +0100 @@ -0,0 +1,9 @@ + + + + + + + + + diff -r 341a0f4b7ce0 -r 5d7c13e998e9 background.js --- a/background.js Sun Nov 04 10:03:05 2018 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,67 +0,0 @@ -/* - * Copyright (C) 2018 Guido Berhoerster - * - * This Source Code Form is subject to the terms of the Mozilla Public - * License, v. 2.0. If a copy of the MPL was not distributed with this - * file, You can obtain one at http://mozilla.org/MPL/2.0/. - */ - -'use strict'; - -var tabsFeeds = new Map(); - -// until content handlers become available to webextensions -// (https://bugzilla.mozilla.org/show_bug.cgi?id=1457500) intercept all -// responses and change the content type from application/atom+xml or -// application/rss+xml to application/xml which will then be handled by a -// content script -browser.webRequest.onHeadersReceived.addListener(details => { - if (details.statusCode != 200 || - typeof details.responseHeaders === 'undefined') { - return; - } - - let contentTypeHeader = details.responseHeaders.find(element => { - return element.name.toLowerCase() === 'content-type'; - }); - if (typeof contentTypeHeader !== 'undefined') { - let contentType = contentTypeHeader.value.split(';'); - let mediaType = contentType[0].trim().toLowerCase(); - if (mediaType === 'application/atom+xml' || - mediaType === 'application/rss+xml') { - contentType[0] = 'application/xml'; - contentTypeHeader.value = contentType.join(';'); - } - } - - return {responseHeaders: details.responseHeaders}; -}, {urls: ['http://*/*', 'https://*/*'], types: ['main_frame']}, -['blocking', 'responseHeaders']); - -browser.runtime.onMessage.addListener((request, sender, sendResponse) => { - let tab = sender.tab; - if (typeof tab !== 'undefined') { - // content script sending feeds - tabsFeeds.set(tab.id, request); - browser.pageAction.show(tab.id); - } else { - let response = tabsFeeds.get(request); - // popup querying feeds - sendResponse(tabsFeeds.get(request)); - } -}); - -browser.tabs.onUpdated.addListener((id, changeInfo, tab) => { - if (typeof changeInfo.url === 'undefined') { - // filter out updates which do not change the URL - return; - } - - // hide the page action when the URL changes since it is no longer valid, - // it will be shown again if the content script detects a feed - browser.pageAction.hide(tab.id); -}); - -browser.tabs.onRemoved.addListener((tabId, removeInfo) => { - tabsFeeds.delete(tabId); -}); diff -r 341a0f4b7ce0 -r 5d7c13e998e9 content_scripts/feed-preview.js --- a/content_scripts/feed-preview.js Sun Nov 04 10:03:05 2018 +0100 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,598 +0,0 @@ -/* - * Copyright (C) 2018 Guido Berhoerster - * - * This Source Code Form is subject to the terms of the Mozilla Public - * License, v. 2.0. If a copy of the MPL was not distributed with this - * file, You can obtain one at http://mozilla.org/MPL/2.0/. - */ - -'use strict'; - -const ALLOWED_PROTOCOLS = new Set(['http:', 'https:', 'ftp:']); - -function encodeXML(str) { - return str.replace(/[<>&'"]/g, c => { - switch (c) { - case '<': return '<'; - case '>': return '>'; - case '&': return '&'; - case '\'': return '''; - case '"': return '"'; - } - }); -} - -function parseDate(s) { - let date = new Date(s); - - return isNaN(date) ? new Date(0) : date; -} - -function parseURL(text, baseURL = window.location.href) { - let url; - - try { - url = new URL(text, baseURL); - } catch (e) { - return null; - } - if (!ALLOWED_PROTOCOLS.has(url.protocol)) { - return null; - } - - return url; -} - -function feedNSResolver(prefix) { - switch (prefix) { - case 'atom': - return 'http://www.w3.org/2005/Atom' - case 'rss': - return 'http://my.netscape.com/rdf/simple/0.9/' - } - return null; -} - -function feedQueryXPath(feedDocument, scopeElement, xpathQuery) { - return feedDocument.evaluate(xpathQuery, scopeElement, feedNSResolver, - XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue; -} - -function feedQueryXPathAll(feedDocument, scopeElement, xpathQuery) { - let result = feedDocument.evaluate(xpathQuery, scopeElement, feedNSResolver, - XPathResult.ORDERED_NODE_ITERATOR_TYPE, null); - let nodes = []; - for (let node = result.iterateNext(); node !== null; - node = result.iterateNext()) { - nodes.push(node); - } - - return nodes; -} - -class FeedLogo { - constructor(url, title = '') { - this.url = url; - this.title = title; - } -} - -class RSS1Logo extends FeedLogo { - constructor(feedDocument, imageElement) { - let urlElement = feedQueryXPath(feedDocument, imageElement, - './rss:url'); - if (urlElement === null) { - throw new TypeError('missing element in element'); - } - let url = parseURL(urlElement.textContent.trim()); - if (url === null) { - throw new TypeError('invalid URL in element'); - } - super(url); - - let titleElement = feedQueryXPath(feedDocument, imageElement, - './rss:title'); - if (titleElement !== null) { - this.title = titleElement.textContent.trim(); - } - } -} - -class RSS2Logo extends FeedLogo { - constructor(feedDocument, imageElement) { - let urlElement = feedQueryXPath(feedDocument, imageElement, './url'); - if (urlElement === null) { - throw new TypeError('missing element in element'); - } - let url = parseURL(urlElement.textContent.trim()); - if (url === null) { - throw new TypeError('invalid URL in element'); - } - super(url); - - let titleElement = feedQueryXPath(feedDocument, imageElement, - './title'); - if (titleElement !== null) { - this.title = titleElement.textContent.trim(); - } - } -} - -class AtomLogo extends FeedLogo { - constructor(logoElement) { - let url = parseURL(logoElement.textContent.trim()); - if (url === null) { - throw new TypeError('invalid URL in element'); - } - super(url); - } -} - -class FeedEntryFile { - constructor(url, type = browser.i18n.getMessage('defaultFileType'), - size = 0) { - this.url = url; - let filename = url.pathname.split('/').pop(); - this.filename = filename !== '' ? filename : - browser.i18n.getMessage('defaultFileName'); - this.type = type; - this.size = size; - } -} - -class RSS2EntryFile extends FeedEntryFile { - constructor(enclosureElement) { - let url = parseURL(enclosureElement.getAttribute('url')); - if (url === null) { - throw new TypeError('invalid URL in element'); - } - super(url); - - let type = enclosureElement.getAttribute('type'); - if (type !== null) { - this.type = type; - } - - let size = parseInt(enclosureElement.getAttribute('length'), 10); - if (!isNaN(size)) { - this.size = size; - } - } -} - -class FeedEntry { - constructor(title = browser.i18n.getMessage('defaultFeedEntryTitle'), - url = null, date = new Date(0), content = '', files = []) { - this.title = title; - this.url = url; - this.date = date; - this._content; - this.content = content; - this.files = files; - } - - set content(content) { - this._content = this.normalizeContent(content); - } - - get content() { - return this._content; - } - - normalizeContent(text) { - let parsedDocument = new DOMParser().parseFromString(text, 'text/html'); - - let linkElement = parsedDocument.createElement('link'); - linkElement.rel = 'stylesheet'; - linkElement.href ='style/entry-content.css'; - parsedDocument.head.appendChild(linkElement); - - return new XMLSerializer().serializeToString(parsedDocument); - } -} - -class RSS1Entry extends FeedEntry { - constructor(feedDocument, itemElement) { - super(); - - let titleElement = feedQueryXPath(feedDocument, itemElement, - './rss:title'); - if (titleElement !== null) { - this.title = titleElement.textContent; - } - - let linkElement = feedQueryXPath(feedDocument, itemElement, - './rss:link'); - if (linkElement !== null) { - this.url = parseURL(linkElement.textContent); - } - } -} - -class RSS2Entry extends FeedEntry { - constructor(feedDocument, itemElement) { - super(); - - let titleElement = feedQueryXPath(feedDocument, itemElement, './title'); - if (titleElement !== null) { - this.title = titleElement.textContent; - } - - let linkElement = feedQueryXPath(feedDocument, itemElement, './link'); - if (linkElement !== null) { - this.url = parseURL(linkElement.textContent); - } - - let pubDateElement = feedQueryXPath(feedDocument, itemElement, - './pubDate'); - if (pubDateElement !== null) { - this.date = parseDate(pubDateElement.textContent); - } - - let descriptionElement = feedQueryXPath(feedDocument, itemElement, - './description'); - if (descriptionElement !== null) { - this.content = descriptionElement.textContent.trim(); - } - - for (let enclosureElement of - feedQueryXPathAll(feedDocument, itemElement, './enclosure')) { - try { - let entryFile = new RSS2EntryFile(enclosureElement); - this.files.push(entryFile); - } catch (e) {} - } - } -} - -class AtomEntry extends FeedEntry { - constructor(feedDocument, entryElement) { - super(); - - let titleElement = feedQueryXPath(feedDocument, entryElement, - './atom:title'); - if (titleElement !== null) { - this.title = titleElement.textContent.trim(); - } - - let linkElement = feedQueryXPath(feedDocument, entryElement, - './atom:link[@href][@rel="alternate"]'); - if (linkElement !== null) { - this.url = parseURL(linkElement.getAttribute('href')); - } - - let updatedElement = feedQueryXPath(feedDocument, entryElement, - './atom:updated'); - if (updatedElement !== null) { - this.date = parseDate(updatedElement.textContent); - } - - let contentElement = feedQueryXPath(feedDocument, entryElement, - './atom:content'); - if (contentElement === null) { - contentElement = feedQueryXPath(feedDocument, entryElement, - './atom:summary'); - } - if (contentElement !== null) { - let contentType = contentElement.getAttribute('type'); - if (contentType === null) { - contentType = 'text'; - } - contentType = contentType.toLowerCase(); - if (contentType === 'xhtml') { - this.content = contentElement.innerHTML; - } else if (contentType === 'html') { - this.content = contentElement.textContent; - } else { - let encodedContent = - encodeXML(contentElement.textContent.trim()); - this.content = `

${encodedContent}

`; - } - } - } -} - -class Feed { - constructor(title = browser.i18n.getMessage('defaultFeedTitle'), - subtitle = '', logo = null, entries = []) { - this.title = title; - this.subtitle = subtitle; - this.logo = logo; - this.entries = entries; - } - - async createPreviewDocument() { - let url = browser.extension.getURL('web_resources/feed-preview.xhtml'); - let response; - let text; - try { - response = await fetch(url); - text = await response.text(); - } catch (e) { - console.log(`Error: failed to read preview template: ${e.message}`); - return; - } - let previewDocument = (new DOMParser()).parseFromString(text, - 'application/xhtml+xml'); - - previewDocument.querySelector('base').href = - browser.extension.getURL('web_resources/'); - - previewDocument.querySelector('title').textContent = this.title; - previewDocument.querySelector('#feed-title').textContent = this.title; - previewDocument.querySelector('#feed-subtitle').textContent = - this.subtitle; - if (this.logo !== null) { - let feedLogoTemplate = - previewDocument.querySelector('#feed-logo-template'); - let logoNode = previewDocument.importNode(feedLogoTemplate.content, - true); - let imgElement = logoNode.querySelector('#feed-logo'); - imgElement.setAttribute('src', this.logo.url); - imgElement.setAttribute('alt', this.logo.title); - previewDocument.querySelector('#feed-header').prepend(logoNode); - } - - let entryTemplateElement = - previewDocument.querySelector('#entry-template'); - let entryTitleTemplateElement = - previewDocument.querySelector('#entry-title-template'); - let entryTitleLinkedTemplateElement = - previewDocument.querySelector('#entry-title-linked-template'); - let entryFileListTemplateElement = - previewDocument.querySelector('#entry-files-list-template'); - let entryFileTemplateElement = - previewDocument.querySelector('#entry-file-template'); - if (this.entries.length === 0) { - let hintTemplateElement = - previewDocument.querySelector('#no-entries-hint-template'); - let hintNode = - previewDocument.importNode(hintTemplateElement.content, - true); - hintNode.querySelector("#no-entries-hint").textContent = - browser.i18n.getMessage('noEntriesHint'); - - previewDocument.body.append(hintNode); - } - for (let entry of this.entries) { - let entryNode = - previewDocument.importNode(entryTemplateElement.content, - true); - let titleElement; - let titleNode; - - if (entry.url !== null) { - titleNode = previewDocument - .importNode(entryTitleLinkedTemplateElement.content, - true); - titleElement = titleNode.querySelector('.entry-link'); - titleElement.href = entry.url; - titleElement.title = entry.title; - } else { - titleNode = previewDocument - .importNode(entryTitleTemplateElement.content, true); - titleElement = titleNode.querySelector('.entry-title'); - } - titleElement.textContent = entry.title; - entryNode.querySelector('.entry-header').prepend(titleNode); - - let timeElement = entryNode.querySelector('.entry-date > time'); - timeElement.textContent = entry.date.toLocaleString(); - - let contentElement = entryNode.querySelector('.entry-content'); - contentElement.srcdoc = entry.content; - contentElement.title = entry.title; - - if (entry.files.length > 0) { - let fileListNode = previewDocument - .importNode(entryFileListTemplateElement.content, true); - fileListNode.querySelector('.entry-files-title').textContent = - browser.i18n.getMessage('filesTitle'); - let fileListElement = - fileListNode.querySelector('.entry-files-list'); - - for (let file of entry.files) { - let fileNode = previewDocument - .importNode(entryFileTemplateElement.content, true); - - let fileLinkElement = - fileNode.querySelector('.entry-file-link'); - fileLinkElement.href = file.url; - fileLinkElement.title = file.filename; - fileLinkElement.textContent = file.filename; - - fileNode.querySelector('.entry-file-info').textContent = - `(${file.type}, ${file.size} bytes)`; - - fileListElement.appendChild(fileNode); - } - - entryNode.querySelector('.entry').append(fileListNode); - } - - previewDocument.body.append(entryNode); - } - - return previewDocument; - } -} - -class RSS1Feed extends Feed { - constructor(feedDocument) { - super(); - - let documentElement = feedDocument.documentElement; - let titleElement = feedQueryXPath(feedDocument, documentElement, - './rss:channel/rss:title'); - if (titleElement !== null) { - this.title = titleElement.textContent; - } - - let descriptionElement = feedQueryXPath(feedDocument, documentElement, - './channel/description'); - if (descriptionElement !== null) { - this.subtitle = descriptionElement.textContent; - } - - let imageElement = feedQueryXPath(feedDocument, documentElement, - './rss:image'); - if (imageElement !== null) { - try { - let logo = new RSS1Logo(feedDocument, imageElement); - this.logo = logo; - } catch (e) {} - } - - let itemElements = feedQueryXPathAll(feedDocument, documentElement, - './rss:item'); - for (let itemElement of itemElements) { - let entry = new RSS1Entry(feedDocument, itemElement); - if (typeof entry !== 'undefined') { - this.entries.push(entry); - } - } - } -} - -class RSS2Feed extends Feed { - constructor(feedDocument) { - super(); - - let documentElement = feedDocument.documentElement; - let titleElement = feedQueryXPath(feedDocument, documentElement, - './channel/title'); - if (titleElement !== null) { - this.title = titleElement.textContent; - } - - let descriptionElement = feedQueryXPath(feedDocument, documentElement, - './channel/description'); - if (descriptionElement !== null) { - this.subtitle = descriptionElement.textContent; - } - - let imageElement = feedQueryXPath(feedDocument, documentElement, - './channel/image'); - if (imageElement !== null) { - try { - let logo = new RSS2Logo(feedDocument, imageElement); - this.logo = logo; - } catch (e) {} - } - - let itemElements = feedQueryXPathAll(feedDocument, documentElement, - './channel/item'); - for (let itemElement of itemElements) { - let entry = new RSS2Entry(feedDocument, itemElement); - if (typeof entry !== 'undefined') { - this.entries.push(entry); - } - } - } -} - -class AtomFeed extends Feed { - constructor(feedDocument, atomVersion) { - super(); - - let documentElement = feedDocument.documentElement; - let titleElement = feedQueryXPath(feedDocument, documentElement, - './atom:title'); - if (titleElement !== null) { - this.title = titleElement.textContent.trim(); - } - - let subtitleElement = feedQueryXPath(feedDocument, documentElement, - './atom:subtitle'); - if (subtitleElement !== null) { - this.subtitle = subtitleElement.textContent.trim(); - } - - let logoElement = feedQueryXPath(feedDocument, documentElement, - './atom:logo'); - if (logoElement !== null) { - try { - let logo = new AtomLogo(logoElement); - this.logo = logo; - } catch (e) {} - } - - let entryElements = feedQueryXPathAll(feedDocument, documentElement, - './atom:entry'); - for (let entryElement of entryElements) { - this.entries.push(new AtomEntry(feedDocument, entryElement)); - } - } -} - -function probeFeedType(feedDocument) { - if (feedDocument.documentElement.nodeName === 'feed') { - let version = feedDocument.documentElement.getAttribute('version'); - if (version === null) { - version = '1.0'; - } - for (let attr of feedDocument.documentElement.attributes) { - if (attr.name === 'xmlns' && - attr.value === 'http://www.w3.org/2005/Atom') { - return ['atom', version]; - } - } - } else if (feedDocument.documentElement.nodeName === 'rss') { - let version = feedDocument.documentElement.getAttribute('version'); - if (version !== null) { - return ['rss', version]; - } - } else if (feedDocument.documentElement.localName.toLowerCase() === 'rdf') { - for (let attr of feedDocument.documentElement.attributes) { - if (attr.name === 'xmlns' && - attr.value === 'http://my.netscape.com/rdf/simple/0.9/') { - return ['rss', '0.9']; - } - } - } - - return [undefined, undefined]; -} - -async function replaceDocumentWithPreview(type, version) { - let feed; - switch (type) { - case 'rss': - switch (version) { - case '0.9': - case '1.0': - feed = new RSS1Feed(document, version); - break; - case '0.90': - case '0.91': - case '0.92': - case '0.93': - case '0.94': - case '2.0': - feed = new RSS2Feed(document, version); - break; - default: - return; - } - break; - case 'atom': - feed = new AtomFeed(document, version); - break; - default: - return; - } - - // replace original document with preview - let previewDocument = await feed.createPreviewDocument(); - if (typeof previewDocument === 'undefined') { - return; - } - let documentElement = previewDocument.documentElement; - document.replaceChild(document.importNode(documentElement, true), - document.documentElement); -} - -let [type, version] = probeFeedType(document); -if (typeof type !== 'undefined') { - replaceDocumentWithPreview(type, version); -} diff -r 341a0f4b7ce0 -r 5d7c13e998e9 js/background.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/js/background.js Thu Nov 08 16:30:34 2018 +0100 @@ -0,0 +1,161 @@ +/* + * Copyright (C) 2018 Guido Berhoerster + * + * This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. + */ + +'use strict'; + +import * as feedParser from './feed-parser.js'; +import {renderFeedPreview} from './feed-preview.js'; + +const FEED_MAGIC = [ + ' response.text()); + +function parseContentType(header) { + let contentType = { + mediaType: '', + charset: 'utf-8' + }; + let parts = header.toLowerCase().split(';'); + contentType.mediaType = parts.shift().trim(); + for (let parameter of parts) { + let [, name, value, ] = parameter.trim().split(/([^=]+)="?([^"]*)"?/); + if (name.toLowerCase() === 'charset') { + contentType.charset = value.toLowerCase(); + break; + } + } + + return contentType; +} + +async function handleFeed(inputText, url) { + // fast-path: eliminate XML documents which cannot be Atom nor RSS feeds + let inputTextStart = inputText.substring(0, 512); + if (!FEED_MAGIC.some(element => inputTextStart.includes(element))) { + return inputText; + } + + let feed; + try { + feed = (new feedParser.FeedParser).parseFromString(inputText, url); + } catch (e) { + if (e instanceof feedParser.ParserError || + e instanceof feedParser.UnsupportedFeedTypeError) { + // let the browser deal with non-well formed XML or XML documents + // which are not supported Atom or RSS feeds + return inputText; + } + throw e; + } + console.log(`parsed feed ${url}:\n`, feed); + + // render the preview document + let feedPreviewDocument = new DOMParser() + .parseFromString(await fetchingFeedPreview, 'text/html'); + renderFeedPreview(feedPreviewDocument, feed); + + return new XMLSerializer().serializeToString(feedPreviewDocument); +} + +browser.webRequest.onHeadersReceived.addListener(details => { + if (details.statusCode !== 200) { + return {}; + } + + let contentTypeIndex = details.responseHeaders.findIndex(header => + header.name.toLowerCase() === 'content-type' && + typeof header.value !== 'undefined'); + if (contentTypeIndex < 0) { + // no Content-Type header found + return {}; + } + let headerValue = details.responseHeaders[contentTypeIndex].value + let contentType = parseContentType(headerValue); + // until content handlers become available to webextensions + // (https://bugzilla.mozilla.org/show_bug.cgi?id=1457500) intercept all + // responses and change the content type from application/atom+xml or + // application/rss+xml to application/xml which will then be probed for + // Atom or RSS content + switch (contentType.mediaType) { + case 'application/atom+xml': + case 'application/rss+xml': + case 'application/rdf+xml': + case 'application/xml': + break; + default: + // non-XML media type + return {}; + } + console.log(`response is an XML document\n`, + `media type: ${contentType.mediaType}\n`, + `charset: ${contentType.charset}`); + + let decoder; + try { + decoder = new TextDecoder(contentType.charset); + } catch (e) { + if (e instanceof RangeError) { + // unsupported charset + return {}; + } else { + throw e; + } + } + let encoder = new TextEncoder(); + let inputText = ''; + let filter = browser.webRequest.filterResponseData(details.requestId); + filter.addEventListener('data', ev => { + inputText += decoder.decode(ev.data, {stream: true}); + }); + filter.addEventListener('stop', async ev => { + let result = await handleFeed(inputText, details.url); + filter.write(encoder.encode(result)); + filter.close(); + }); + + details.responseHeaders[contentTypeIndex] = { + name: 'Content-Type', + value: `application/xml;charset=${contentType.charset}` + }; + + return {responseHeaders: details.responseHeaders}; +}, + {urls: ['http://*/*', 'https://*/*'], types: ['main_frame']}, + ['blocking', 'responseHeaders']); + +browser.runtime.onMessage.addListener((request, sender, sendResponse) => { + let tab = sender.tab; + if (typeof tab !== 'undefined') { + // content script sending feeds + tabsFeeds.set(tab.id, request); + browser.pageAction.show(tab.id); + } else { + // popup querying feeds + sendResponse(tabsFeeds.get(request)); + } +}); + +browser.tabs.onUpdated.addListener((id, changeInfo, tab) => { + if (typeof changeInfo.url === 'undefined') { + // filter out updates which do not change the URL + return; + } + + // hide the page action when the URL changes since it is no longer valid, + // it will be shown again if the content script detects a feed + browser.pageAction.hide(tab.id); +}); + +browser.tabs.onRemoved.addListener((tabId, removeInfo) => { + tabsFeeds.delete(tabId); +}); diff -r 341a0f4b7ce0 -r 5d7c13e998e9 js/feed-parser.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/js/feed-parser.js Thu Nov 08 16:30:34 2018 +0100 @@ -0,0 +1,538 @@ +/* + * Copyright (C) 2018 Guido Berhoerster + * + * This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. + */ + +'use strict'; + +export const XMLNS = { + ATOM10: 'http://www.w3.org/2005/Atom', + RSS09: 'http://my.netscape.com/rdf/simple/0.9/' +} +const ALLOWED_LINK_PROTOCOLS = new Set(['http:', 'https:', 'ftp:']); + +function encodeXML(str) { + return str.replace(/[<>&'"]/g, c => { + switch (c) { + case '<': return '<'; + case '>': return '>'; + case '&': return '&'; + case '\'': return '''; + case '"': return '"'; + } + }); +} + +function parseDate(s) { + let date = new Date(s); + + return isNaN(date) ? new Date(0) : date; +} + +function parseURL(text, baseURL = '') { + let url; + + try { + url = new URL(text, baseURL); + } catch (e) { + return null; + } + if (!ALLOWED_LINK_PROTOCOLS.has(url.protocol)) { + return null; + } + + return url; +} + +function feedNSResolver(prefix) { + switch (prefix) { + case 'atom': + return XMLNS.ATOM10; + case 'rss': + return XMLNS.RSS09; + } + return null; +} + +function feedQueryXPath(feedDocument, scopeElement, xpathQuery) { + return feedDocument.evaluate(xpathQuery, scopeElement, feedNSResolver, + XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue; +} + +function feedQueryXPathAll(feedDocument, scopeElement, xpathQuery) { + let result = feedDocument.evaluate(xpathQuery, scopeElement, feedNSResolver, + XPathResult.ORDERED_NODE_ITERATOR_TYPE, null); + let nodes = []; + for (let node = result.iterateNext(); node !== null; + node = result.iterateNext()) { + nodes.push(node); + } + + return nodes; +} + +export class ParserError extends Error { + constructor() { + super(...arguments); + this.name = this.constructor.name; + } +} + +export class UnsupportedFeedTypeError extends Error { + constructor(message = 'Document is not a supported feed', ...params) { + super(message, ...params); + this.name = this.constructor.name; + } +} + +export class ProtocolError extends Error { + constructor(url, status, statusText, ...params) { + let message = `Protocol error: Transfer of ${url} failed with: ` + + `${status} ${statusText}` + super(message, ...params); + this.name = this.constructor.name; + this.url = url; + this.status = status; + this.statusText = statusText; + } +} + +class FeedLogo { + constructor(url, {title = ''} = {}) { + this.url = url; + this.title = title; + } +} + +class FeedEntryFile { + constructor(url, {type = browser.i18n.getMessage('defaultFileType'), + size = 0} = {}) { + this.filename = undefined; + this._url = undefined; + this.url = url; + this.type = type; + this.size = size; + } + + set url(url) { + this._url = url; + let filename = url.pathname.split('/').pop(); + this.filename = filename !== '' ? filename : + browser.i18n.getMessage('defaultFileName'); + } + + get url() { + return this._url; + } +} + +class FeedEntry { + constructor({title = browser.i18n.getMessage('defaultFeedEntryTitle'), + link = undefined, date = new Date(0), content = '', + files = []} = {}) { + this.title = title; + this.link = link; + this.date = date; + this._content = undefined; + this.content = content; + this.files = files; + } + + normalizeContent(text) { + if (typeof text === 'undefined') { + return + } + + let contentDocument = document.implementation.createHTMLDocument(); + let parsedDocument = new DOMParser().parseFromString(text, 'text/html'); + contentDocument.body = contentDocument.adoptNode(parsedDocument.body); + return new XMLSerializer().serializeToString(contentDocument); + } + + set content(content) { + this._content = this.normalizeContent(content); + } + + get content() { + return this._content; + } +} + +class Feed { + constructor(url, {title = browser.i18n.getMessage('defaultFeedTitle'), + subtitle = '', logo, entries = []} = {}) { + this.url = url; + this.title = title; + this.subtitle = subtitle; + this.logo = logo; + this.entries = entries; + } +} + +export class FeedParser { + static probeFeed(feedDocument) { + let documentElement = feedDocument.documentElement; + if (documentElement.nodeName === 'feed' && + documentElement.namespaceURI === XMLNS.ATOM10) { + let version = documentElement.getAttribute('version'); + if (version === null) { + version = '1.0'; + } + if (version === '1.0') { + return ['atom', version]; + } + } else if (documentElement.nodeName === 'rss') { + let version = documentElement.getAttribute('version'); + switch (version) { + case '0.90': + case '0.91': + case '0.92': + case '0.93': + case '0.94': + case '2.0': + return ['rss', version]; + } + } else if (documentElement.localName.toLowerCase() === 'rdf' && + documentElement.getAttribute('xmlns') === XMLNS.RSS09) { + return ['rss', '0.9']; + } + + return [undefined, undefined]; + } + + constructor() { + this.url = undefined; + this.document = undefined; + } + + parseAtomLogo(logoElement) { + let url = parseURL(logoElement.textContent.trim(), this.url); + if (url === null) { + throw new TypeError('invalid URL in element'); + } + return new FeedLogo(url); + } + + parseAtomEntry(entryElement) { + let title; + let link; + let date; + let content; + let titleElement = feedQueryXPath(this.document, entryElement, + './atom:title'); + if (titleElement !== null) { + title = titleElement.textContent.trim(); + } + + let linkElement = feedQueryXPath(this.document, entryElement, + './atom:link[@href][@rel="alternate"]'); + if (linkElement !== null) { + link = parseURL(linkElement.getAttribute('href'), this.url); + } + + let updatedElement = feedQueryXPath(this.document, entryElement, + './atom:updated'); + if (updatedElement !== null) { + date = parseDate(updatedElement.textContent); + } + + let contentElement = feedQueryXPath(this.document, entryElement, + './atom:content'); + if (contentElement === null) { + contentElement = feedQueryXPath(this.document, entryElement, + './atom:summary'); + } + if (contentElement !== null) { + let contentType = contentElement.getAttribute('type'); + if (contentType === null) { + contentType = 'text'; + } + contentType = contentType.toLowerCase(); + if (contentType === 'xhtml') { + content = contentElement.innerHTML; + } else if (contentType === 'html') { + content = contentElement.textContent; + } else { + let encodedContent = + encodeXML(contentElement.textContent.trim()); + content = `

${encodedContent}

`; + } + } + + return new FeedEntry({title, link, date, content}); + } + + parseAtomFeed() { + let title; + let subtitle; + let logo; + let entries = []; + let documentElement = this.document.documentElement; + + let titleElement = feedQueryXPath(this.document, documentElement, + './atom:title'); + if (titleElement !== null) { + title = titleElement.textContent.trim(); + } + + let subtitleElement = feedQueryXPath(this.document, documentElement, + './atom:subtitle'); + if (subtitleElement !== null) { + subtitle = subtitleElement.textContent.trim(); + } + + let logoElement = feedQueryXPath(this.document, documentElement, + './atom:logo'); + if (logoElement !== null) { + try { + logo = this.parseAtomLogo(logoElement); + } catch (e) { + if (!(e instanceof TypeError)) { + throw e; + } + } + } + + let entryElements = feedQueryXPathAll(this.document, documentElement, + './atom:entry'); + for (let entryElement of entryElements) { + entries.push(this.parseAtomEntry(entryElement)); + } + + return new Feed(this.url, {title, subtitle, logo, entries}); + } + + parseRSS1Logo(imageElement) { + let title; + let urlElement = feedQueryXPath(this.document, imageElement, + './rss:url'); + if (urlElement === null) { + throw new TypeError('missing element in element'); + } + let url = parseURL(urlElement.textContent.trim(), this.url); + if (url === null) { + throw new TypeError('invalid URL in element'); + } + + let titleElement = feedQueryXPath(this.document, imageElement, + './rss:title'); + if (titleElement !== null) { + title = titleElement.textContent.trim(); + } + + return new FeedLogo(url, {title}); + } + + parseRSS1Entry(itemElement) { + let title; + let link; + let titleElement = feedQueryXPath(this.document, itemElement, + './rss:title'); + if (titleElement !== null) { + title = titleElement.textContent; + } + + let linkElement = feedQueryXPath(this.document, itemElement, + './rss:link'); + if (linkElement !== null) { + link = parseURL(linkElement.textContent, this.url); + } + + return new FeedEntry({title, link}); + } + + parseRSS1Feed() { + let title; + let subtitle; + let logo; + let entries = []; + let documentElement = this.document.documentElement; + let titleElement = feedQueryXPath(this.document, documentElement, + './rss:channel/rss:title'); + if (titleElement !== null) { + title = titleElement.textContent; + } + + let descriptionElement = feedQueryXPath(this.document, documentElement, + './channel/description'); + if (descriptionElement !== null) { + subtitle = descriptionElement.textContent; + } + + let imageElement = feedQueryXPath(this.document, documentElement, + './rss:image'); + if (imageElement !== null) { + try { + logo = this.parseRSS1Logo(imageElement); + } catch (e) { + if (!(e instanceof TypeError)) { + throw e; + } + } + } + + let itemElements = feedQueryXPathAll(this.document, documentElement, + './rss:item'); + for (let itemElement of itemElements) { + let entry = this.parseRSS1Entry(itemElement); + if (typeof entry !== 'undefined') { + entries.push(entry); + } + } + + return new Feed(this.url, {title, subtitle, logo, entries}); + } + + parseRSS2Logo(imageElement) { + let title; + let urlElement = feedQueryXPath(this.document, imageElement, './url'); + if (urlElement === null) { + throw new TypeError('missing element in element'); + } + let url = parseURL(urlElement.textContent.trim(), this.url); + if (url === null) { + throw new TypeError('invalid URL in element'); + } + + let titleElement = feedQueryXPath(this.document, imageElement, + './title'); + if (titleElement !== null) { + title = titleElement.textContent.trim(); + } + + return new FeedLogo(url, {title}); + } + + parseRSS2EntryFile(enclosureElement) { + let type; + let size; + let url = parseURL(enclosureElement.getAttribute('url'), this.url); + if (url === null) { + throw new TypeError('invalid URL in element'); + } + + let typeAttribute = enclosureElement.getAttribute('type'); + if (typeAttribute !== null) { + type = typeAttribute; + } + + let length = parseInt(enclosureElement.getAttribute('length'), + 10); + if (!isNaN(length)) { + size = length; + } + + return new FeedEntryFile(url, {type, size}); + } + + parseRSS2Entry(itemElement) { + let title; + let link; + let date; + let content; + let files = []; + let titleElement = feedQueryXPath(this.document, itemElement, + './title'); + if (titleElement !== null) { + title = titleElement.textContent; + } + + let linkElement = feedQueryXPath(this.document, itemElement, './link'); + if (linkElement !== null) { + link = parseURL(linkElement.textContent, this.url); + } + + let pubDateElement = feedQueryXPath(this.document, itemElement, + './pubDate'); + if (pubDateElement !== null) { + date = parseDate(pubDateElement.textContent); + } + + let descriptionElement = feedQueryXPath(this.document, itemElement, + './description'); + if (descriptionElement !== null) { + content = descriptionElement.textContent.trim(); + } + + for (let enclosureElement of + feedQueryXPathAll(this.document, itemElement, './enclosure')) { + try { + let entryFile = this.parseRSS2EntryFile(enclosureElement); + files.push(entryFile); + } catch (e) { + if (!(e instanceof TypeError)) { + throw e; + } + } + } + + return new FeedEntry({title, link, date, content, files}); + } + + parseRSS2Feed() { + let title; + let subtitle; + let logo; + let entries = []; + let documentElement = this.document.documentElement; + let titleElement = feedQueryXPath(this.document, documentElement, + './channel/title'); + if (titleElement !== null) { + title = titleElement.textContent; + } + + let descriptionElement = feedQueryXPath(this.document, documentElement, + './channel/description'); + if (descriptionElement !== null) { + subtitle = descriptionElement.textContent; + } + + let imageElement = feedQueryXPath(this.document, documentElement, + './channel/image'); + if (imageElement !== null) { + try { + logo = this.parseRSS2Logo(imageElement); + } catch (e) { + if (!(e instanceof TypeError)) { + throw e; + } + } + } + + let itemElements = feedQueryXPathAll(this.document, documentElement, + './channel/item'); + for (let itemElement of itemElements) { + let entry = this.parseRSS2Entry(itemElement); + if (typeof entry !== 'undefined') { + entries.push(entry); + } + } + + return new Feed(this.url, {title, subtitle, logo, entries}); + } + + parseFromString(xmlString, url) { + this.url = url; + this.document = new DOMParser().parseFromString(xmlString, + 'application/xml'); + if (this.document.documentElement.nodeName.toLowerCase() === + 'parsererror') { + throw new ParserError(this.document.documentElement.textContent); + } + + let [type, version] = this.constructor.probeFeed(this.document); + if (type === 'atom') { + return this.parseAtomFeed(); + } else if (type === 'rss') { + if (version === '0.9') { + return this.parseRSS1Feed(); + } else { + return this.parseRSS2Feed(); + } + } + throw new UnsupportedFeedTypeError(); + } +} diff -r 341a0f4b7ce0 -r 5d7c13e998e9 js/feed-preview.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/js/feed-preview.js Thu Nov 08 16:30:34 2018 +0100 @@ -0,0 +1,124 @@ +/* + * Copyright (C) 2018 Guido Berhoerster + * + * This Source Code Form is subject to the terms of the Mozilla Public + * License, v. 2.0. If a copy of the MPL was not distributed with this + * file, You can obtain one at http://mozilla.org/MPL/2.0/. + */ + +'use strict'; + +export function renderFeedPreview(feedPreviewDocument, feed) { + // inject XSL stylesheet which transforms XHTML to HTML allowing the use of + // the HTML DOM + let xslFilename = browser.runtime.getURL('web_resources/xhtml-to-html.xsl'); + let xmlStylesheetNode = + feedPreviewDocument.createProcessingInstruction('xml-stylesheet', + `type="application/xslt+xml" href="${xslFilename}"`); + feedPreviewDocument.firstChild.after(xmlStylesheetNode); + + feedPreviewDocument.querySelector('link[rel=stylesheet]').href = + browser.runtime.getURL('web_resources/style/feed-preview.css'); + + feedPreviewDocument.querySelector('title').textContent = feed.title; + feedPreviewDocument.querySelector('#feed-title').textContent = feed.title; + feedPreviewDocument.querySelector('#feed-subtitle').textContent = + feed.subtitle; + + if (typeof feed.logo !== 'undefined') { + let feedLogoTemplate = + feedPreviewDocument.querySelector('#feed-logo-template'); + let logoNode = feedPreviewDocument.importNode(feedLogoTemplate.content, + true); + let imgElement = logoNode.querySelector('#feed-logo'); + imgElement.setAttribute('src', feed.logo.url); + imgElement.setAttribute('alt', feed.logo.title); + feedPreviewDocument.querySelector('#feed-header').prepend(logoNode); + } + + let entryTemplateElement = + feedPreviewDocument.querySelector('#entry-template'); + let entryTitleTemplateElement = + feedPreviewDocument.querySelector('#entry-title-template'); + let entryTitleLinkedTemplateElement = + feedPreviewDocument.querySelector('#entry-title-linked-template'); + let entryFileListTemplateElement = + feedPreviewDocument.querySelector('#entry-files-list-template'); + let entryFileTemplateElement = + feedPreviewDocument.querySelector('#entry-file-template'); + if (feed.entries.length === 0) { + let hintTemplateElement = + previewDocument.querySelector('#no-entries-hint-template'); + let hintNode = previewDocument.importNode(hintTemplateElement.content, + true); + hintNode.querySelector("#no-entries-hint").textContent = + browser.i18n.getMessage('noEntriesHint'); + + previewDocument.body.append(hintNode); + } + for (let entry of feed.entries) { + let entryNode = + feedPreviewDocument.importNode(entryTemplateElement.content, + true); + let titleElement; + let titleNode; + + if (typeof entry.link !== 'undefined') { + titleNode = feedPreviewDocument + .importNode(entryTitleLinkedTemplateElement.content, true); + titleElement = titleNode.querySelector('.entry-link'); + titleElement.href = entry.link; + titleElement.title = entry.title; + } else { + titleNode = feedPreviewDocument + .importNode(entryTitleTemplateElement.content, true); + titleElement = titleNode.querySelector('.entry-title'); + } + titleElement.textContent = entry.title; + entryNode.querySelector('.entry-header').prepend(titleNode); + + let timeElement = entryNode.querySelector('.entry-date > time'); + timeElement.textContent = entry.date.toLocaleString(); + + let contentElement = entryNode.querySelector('.entry-content'); + let contentDocument = new DOMParser().parseFromString(entry.content, + 'text/html'); + let stylesheetElement = contentDocument.createElement('link'); + stylesheetElement.rel = 'stylesheet'; + stylesheetElement.href = + browser.runtime.getURL('web_resources/style/entry-content.css'); + contentDocument.head.appendChild(stylesheetElement); + contentElement.srcdoc = new XMLSerializer() + .serializeToString(contentDocument); + contentElement.title = entry.title; + + if (entry.files.length > 0) { + let fileListNode = feedPreviewDocument + .importNode(entryFileListTemplateElement.content, true); + fileListNode.querySelector('.entry-files-title').textContent = + browser.i18n.getMessage('filesTitle'); + let fileListElement = + fileListNode.querySelector('.entry-files-list'); + + for (let file of entry.files) { + let fileNode = feedPreviewDocument + .importNode(entryFileTemplateElement.content, true); + + let fileLinkElement = + fileNode.querySelector('.entry-file-link'); + fileLinkElement.href = file.url; + fileLinkElement.title = file.filename; + fileLinkElement.textContent = file.filename; + + fileNode.querySelector('.entry-file-info').textContent = + `(${file.type}, ${file.size} bytes)`; + + fileListElement.appendChild(fileNode); + } + + entryNode.querySelector('.entry').append(fileListNode); + } + + feedPreviewDocument.body.append(entryNode); + } +} diff -r 341a0f4b7ce0 -r 5d7c13e998e9 manifest.json.in --- a/manifest.json.in Sun Nov 04 10:03:05 2018 +0100 +++ b/manifest.json.in Thu Nov 08 16:30:34 2018 +0100 @@ -24,24 +24,18 @@ "webRequestBlocking" ], "background": { - "scripts": [ "background.js" ] + "page": "background.html" }, "content_scripts": [ { "matches": [ "http://*/*", "https://*/*", "file:///*" ], - "js": [ - "content_scripts/feed-probe.js", - "content_scripts/feed-preview.js" - ] + "js": [ "content_scripts/feed-probe.js" ] } ], "web_accessible_resources": [ - "web_resources/feed-preview.xhtml", - "web_resources/arrow.svg", - "web_resources/style/common.css", - "web_resources/style/entry-content.css", - "web_resources/style/feed-preview.css", - "web_resources/style/photon-colors.css" + "web_resources/xhtml-to-html.xsl", + "web_resources/images/arrow.svg", + "web_resources/style/*.css" ], "page_action": { "browser_style": true, diff -r 341a0f4b7ce0 -r 5d7c13e998e9 web_resources/feed-preview.xhtml --- a/web_resources/feed-preview.xhtml Sun Nov 04 10:03:05 2018 +0100 +++ b/web_resources/feed-preview.xhtml Thu Nov 08 16:30:34 2018 +0100 @@ -1,3 +1,5 @@ + + @@ -9,50 +11,49 @@ file, You can obtain one at http://mozilla.org/MPL/2.0/. --> - - + - - + + + + + +

diff -r 341a0f4b7ce0 -r 5d7c13e998e9 web_resources/xhtml-to-html.xsl --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/web_resources/xhtml-to-html.xsl Thu Nov 08 16:30:34 2018 +0100 @@ -0,0 +1,17 @@ + + + + + + + + + +