Merge branch 'v3' into toc-links

2025-05-09 19:29:34 +08:00 · 2019-01-11 19:29:25 +02:00 · 2019-01-11 19:29:25 +02:00 · 37a710fb7a
commit 37a710fb7a
parent 3658a49abc ebe3a3e574
14 changed files with 755 additions and 326 deletions
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@ -1,3 +1,5 @@
 ## Contributor License Agreement

-* Please sign the [Contributor License Agreement](https://docs.google.com/a/owlglobal.io/forms/d/1PfTjEAi67-x0JOTU45SDonJnWy1fWB_J1aopGss34bY/viewform) in order to have your changes merged.
+[![CLA assistant](https://cla-assistant.io/readme/badge/unidoc/unidoc)](https://cla-assistant.io/unidoc/unidoc)
+
+All contributors must sign a contributor license agreement before their code will be reviewed and merged.
--- a/README.md
+++ b/README.md
@ -1,3 +1,26 @@
+# Version 3 - Upcoming
+
+Version 3 of UniDoc is currently in alpha. It marks multiple significant new major advancements as well as many fixes and enhancements:
+
+- [ ] Composite fonts supported and font handling has and is being completely revamped, including unicode support.
+- [ ] Digital signing validation and signing
+- [x] Append mode
+- [x] PDF compression and optimization of outputs with several options 1) combining duplicates, 2) compressed object streams, 3) image points per inch threshold, 4) image quality.
+- [ ] Text extraction significantly improved in quality and support for vectorized (position-based) text extraction (XY)
+- [x] Paragraph in creator handling multiple styles within the same paragraph
+- [x] [Invoice component for easy PDF invoice generation](https://unidoc.io/news/simple-invoices)
+- [x] Table of contents automatically generated
+- [x] Encryption support refactored and AESv3 support added
+- [x] Form field filling and form flattening with appearance generation
+- [x] Getting form field values and listing
+- [x] FDF merge
+
+Go give it a spin, checkout the `v3` branch of unidoc and `v3` branch of unidoc-examples:
+- https://github.com/unidoc/unidoc/tree/v3
+- https://github.com/unidoc/unidoc-examples/tree/v3
+
+---
+
 # UniDoc

 [UniDoc](http://unidoc.io) is a powerful PDF library for Go (golang). The library is written and supported by the owners of the [FoxyUtils.com](https://foxyutils.com) website, where the library is used to power many of the PDF services offered. 
@ -10,25 +33,40 @@
 go get github.com/unidoc/unidoc/...
 ~~~

-## Getting Rid of the Watermark - Get a License
-Out of the box - unidoc is unlicensed and outputs a watermark on all pages, perfect for prototyping.
-To use unidoc in your projects, you need to get a license. We have 3 license types:
+## Features
+UniDoc has a powerful set of features both for reading, processing and writing PDF.
+The following list describes some of the main features:

-* Community: For open source AGPLv3 projects
-* Business Individual
-* Business Unlimited
+- [x] Create PDF reports with easy interface
+- [x] Create PDF invoices (v3)
+- [x] Merge PDF pages
+- [x] Merge page contents
+- [x] Split PDF pages and change page order
+- [x] Rotate pages
+- [x] Extract text from PDF files
+- [x] Extract images
+- [x] Add images to pages
+- [x] Compress and optimize PDF output (v3)
+- [x] Draw watermark on PDF files
+- [x] Advanced page manipulation (blocks/templates)
+- [x] Load PDF templates and modify
+- [x] Flatten forms and generate appearance streams (v3)
+- [x] Fill out forms and FDF merging (v3)
+- [x] Unlock PDF files / remove password
+- [x] Protect PDF files with a password
+- [ ] Digital signatures (v3)

-Get your license on [https://unidoc.io](https://unidoc.io).

-To load your license, simply do:
-```
-unidocLicenseKey := "... your license here ..."
-err := license.SetLicenseKey(unidocLicenseKey)
-if err != nil {
-    fmt.Printf("Error loading license: %v\n", err)
-    os.Exit(1)
-}
-```
+## How can I convince myself and my boss to buy unidoc rather using a free alternative?
+
+The choice is yours. There are multiple respectable efforts out there that can do many good things.
+
+In unidoc, we work hard to provide production quality builds taking every detail into consideration and providing excellent support to our customers.  See our [testimonials](https://unidoc.io) for example.
+
+Security.  We take security very seriously and we restrict access to github.com/unidoc/unidoc repository with protected branches and only 2 of the founders have access and every commit is reviewed prior to being accepted.
+
+The profits are invested back into making unidoc better. We want to make the best possible product and in order to do that we need the best people to contribute. A large fraction of the profits made goes back into developing unidoc.  That way we have been able to get many excellent people to work and contribute to unidoc that would not be able to contribute their work for free.
+

 ## Examples

@ -41,6 +79,20 @@ Contact us if you need any specific examples.
 For reliability, we recommend using specific versions and the vendoring capability of golang.
 Check out the Releases section to see the tagged releases.

+
+## Contributing
+
+[![CLA assistant](https://cla-assistant.io/readme/badge/unidoc/unidoc)](https://cla-assistant.io/unidoc/unidoc)
+
+All contributors must sign a contributor license agreement before their code will be reviewed and merged.
+
+## Support and consulting
+
+Please email us at support@unidoc.io for any queries.
+
+If you have any specific tasks that need to be done, we offer consulting in certain cases.
+Please contact us with a brief summary of what you need and we will get back to you with a quote, if appropriate.
+
 ## Licensing Information

 This library (UniDoc) has a dual license, a commercial one suitable for closed source projects and an
@ -62,16 +114,21 @@ These activities include:
 Please see [pricing](http://unidoc.io/pricing) to purchase a commercial license or contact sales at sales@unidoc.io
 for more info.

-## Contributing
+## Getting Rid of the Watermark - Get a License
+Out of the box - unidoc is unlicensed and outputs a watermark on all pages, perfect for prototyping.
+To use unidoc in your projects, you need to get a license.

-Contributors need to approve the [Contributor License Agreement](https://docs.google.com/a/owlglobal.io/forms/d/1PfTjEAi67-x0JOTU45SDonJnWy1fWB_J1aopGss34bY/viewform) before any code will be reviewed. Preferably add a test case to make sure there is no regression and that the new behaviour is as expected.
+Get your license on [https://unidoc.io](https://unidoc.io).

-## Support and consulting
-
-Please email us at support@unidoc.io for any queries.
-
-If you have any specific tasks that need to be done, we offer consulting in certain cases.
-Please contact us with a brief summary of what you need and we will get back to you with a quote, if appropriate.
+To load your license, simply do:
+```go
+unidocLicenseKey := "... your license here ..."
+err := license.SetLicenseKey(unidocLicenseKey)
+if err != nil {
+    fmt.Printf("Error loading license: %v\n", err)
+    os.Exit(1)
+}
+```


 [contributing]: CONTRIBUTING.md
--- a/codecov.yml
+++ b/codecov.yml
@ -0,0 +1,8 @@
+coverage:
+  status:
+    project:
+      default:
+        target: 35%
+        threshold: 1%
+    patch: false
+    changes: false
--- a/common/version.go
+++ b/common/version.go
@ -10,13 +10,13 @@ import (
 	"time"
 )

-const releaseYear = 2018
-const releaseMonth = 8
-const releaseDay = 14
-const releaseHour = 19
-const releaseMin = 40
+const releaseYear = 2019
+const releaseMonth = 1
+const releaseDay = 10
+const releaseHour = 22
+const releaseMin = 42

 // Version holds version information, when bumping this make sure to bump the released at stamp also.
-const Version = "2.1.1"
+const Version = "3.0.0-alpha.1"

 var ReleasedAt = time.Date(releaseYear, releaseMonth, releaseDay, releaseHour, releaseMin, 0, 0, time.UTC)
--- a/pdf/contentstream/encoding.go
+++ b/pdf/contentstream/encoding.go
@ -54,19 +54,25 @@ func newEncoderFromInlineImage(inlineImage *ContentStreamInlineImage) (core.Stre
 		}
 	}

-	if *filterName == "AHx" {
+	// From Table 94 p. 224 (PDF32000_2008):
+	// Additional Abbreviations in an Inline Image Object:
+
+	switch *filterName {
+	case "AHx", "ASCIIHexDecode":
 		return core.NewASCIIHexEncoder(), nil
-	} else if *filterName == "A85" {
+	case "A85", "ASCII85Decode":
 		return core.NewASCII85Encoder(), nil
-	} else if *filterName == "DCT" {
+	case "DCT", "DCTDecode":
 		return newDCTEncoderFromInlineImage(inlineImage)
-	} else if *filterName == "Fl" {
+	case "Fl", "FlateDecode":
 		return newFlateEncoderFromInlineImage(inlineImage, nil)
-	} else if *filterName == "LZW" {
+	case "LZW", "LZWDecode":
 		return newLZWEncoderFromInlineImage(inlineImage, nil)
-	} else if *filterName == "CCF" {
+	case "CCF", "CCITTFaxDecode":
 		return core.NewCCITTFaxEncoder(), nil
-	} else {
+	case "RL", "RunLengthDecode":
+		return core.NewRunLengthEncoder(), nil
+	default:
 		common.Log.Debug("Unsupported inline image encoding filter name : %s", *filterName)
 		return nil, errors.New("unsupported inline encoding method")
 	}
--- a/pdf/contentstream/inline-image.go
+++ b/pdf/contentstream/inline-image.go
@ -173,7 +173,7 @@ func (img *ContentStreamInlineImage) GetColorSpace(resources *model.PdfPageResou
 		return model.NewPdfColorspaceDeviceRGB(), nil
 	} else if *name == "CMYK" || *name == "DeviceCMYK" {
 		return model.NewPdfColorspaceDeviceCMYK(), nil
-	} else if *name == "I" {
+	} else if *name == "I" || *name == "Indexed" {
 		return nil, errors.New("unsupported Index colorspace")
 	} else {
 		if resources.ColorSpace == nil {
@ -327,27 +327,38 @@ func (csp *ContentStreamParser) ParseInlineImage() (*ContentStreamInlineImage, e
 				return nil, fmt.Errorf("not expecting an operand")
 			}

-			if *param == "BPC" || *param == "BitsPerComponent" {
+			// From 8.9.7 "Inline Images" p. 223 (PDF32000_2008):
+			// The key-value pairs appearing between the BI and ID operators are analogous to those in the dictionary
+			// portion of an image XObject (though the syntax is different).
+			// Table 93 shows the entries that are valid for an inline image, all of which shall have the same meanings
+			// as in a stream dictionary (see Table 5) or an image dictionary (see Table 89).
+			// Entries other than those listed shall be ignored; in particular, the Type, Subtype, and Length
+			// entries normally found in a stream or image dictionary are unnecessary.
+			// For convenience, the abbreviations shown in the table may be used in place of the fully spelled-out keys.
+			// Table 94 shows additional abbreviations that can be used for the names of colour spaces and filters.
+
+			switch *param {
+			case "BPC", "BitsPerComponent":
 				im.BitsPerComponent = valueObj
-			} else if *param == "CS" || *param == "ColorSpace" {
+			case "CS", "ColorSpace":
 				im.ColorSpace = valueObj
-			} else if *param == "D" || *param == "Decode" {
+			case "D", "Decode":
 				im.Decode = valueObj
-			} else if *param == "DP" || *param == "DecodeParms" {
+			case "DP", "DecodeParms":
 				im.DecodeParms = valueObj
-			} else if *param == "F" || *param == "Filter" {
+			case "F", "Filter":
 				im.Filter = valueObj
-			} else if *param == "H" || *param == "Height" {
+			case "H", "Height":
 				im.Height = valueObj
-			} else if *param == "IM" {
+			case "IM", "ImageMask":
 				im.ImageMask = valueObj
-			} else if *param == "Intent" {
+			case "Intent":
 				im.Intent = valueObj
-			} else if *param == "I" {
+			case "I", "Interpolate":
 				im.Interpolate = valueObj
-			} else if *param == "W" || *param == "Width" {
+			case "W", "Width":
 				im.Width = valueObj
-			} else {
+			default:
 				return nil, fmt.Errorf("unknown inline image parameter %s", *param)
 			}
 		}
--- a/pdf/core/primitives.go
+++ b/pdf/core/primitives.go
@ -654,7 +654,11 @@ func (d *PdfObjectDictionary) GetString(key PdfObjectName) (string, bool) {
 }

 // Keys returns the list of keys in the dictionary.
+// If `d` is nil returns a nil slice.
 func (d *PdfObjectDictionary) Keys() []PdfObjectName {
+	if d == nil {
+		return nil
+	}
 	return d.keys
 }

@ -799,7 +803,7 @@ func TraceToDirectObject(obj PdfObject) PdfObject {
 		iobj, isIndirectObj = obj.(*PdfIndirectObject)
 		depth++
 		if depth > TraceMaxDepth {
-			common.Log.Error("Trace depth level beyond 20 - error!")
+			common.Log.Error("ERROR: Trace depth level beyond %d - not going deeper!", TraceMaxDepth)
 			return nil
 		}
 	}
--- a/pdf/fjson/doc.go
+++ b/pdf/fjson/doc.go
@ -0,0 +1,7 @@
+/*
+ * This file is subject to the terms and conditions defined in
+ * file 'LICENSE.md', which is part of this source code package.
+ */
+
+// Package fjson provides support for loading PDF form field data from JSON data/files.
+package fjson
--- a/pdf/fjson/fielddata.go
+++ b/pdf/fjson/fielddata.go
@ -0,0 +1,153 @@
+/*
+ * This file is subject to the terms and conditions defined in
+ * file 'LICENSE.md', which is part of this source code package.
+ */
+
+package fjson
+
+import (
+	"encoding/json"
+	"io"
+	"os"
+
+	"github.com/unidoc/unidoc/pdf/core"
+	"github.com/unidoc/unidoc/pdf/model"
+)
+
+// FieldData represents form field data loaded from JSON file.
+type FieldData struct {
+	values []fieldValue
+}
+
+// fieldValue represents a field name and value for a PDF form field.
+type fieldValue struct {
+	Name  string `json:"name"`
+	Value string `json:"value"`
+
+	// Options lists allowed values if present.
+	Options []string `json:"options,omitempty"`
+}
+
+// LoadFromJSON loads JSON form data from `r`.
+func LoadFromJSON(r io.Reader) (*FieldData, error) {
+	var fdata FieldData
+	err := json.NewDecoder(r).Decode(&fdata.values)
+	if err != nil {
+		return nil, err
+	}
+	return &fdata, nil
+}
+
+// LoadFromJSONFile loads form field data from a JSON file.
+func LoadFromJSONFile(filePath string) (*FieldData, error) {
+	f, err := os.Open(filePath)
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+
+	return LoadFromJSON(f)
+}
+
+// LoadFromPDF loads form field data from a PDF.
+func LoadFromPDF(rs io.ReadSeeker) (*FieldData, error) {
+	pdfReader, err := model.NewPdfReader(rs)
+	if err != nil {
+		return nil, err
+	}
+
+	if pdfReader.AcroForm == nil {
+		return nil, nil
+	}
+
+	var fieldvals []fieldValue
+	fields := pdfReader.AcroForm.AllFields()
+	for _, f := range fields {
+		var options []string
+		optMap := make(map[string]struct{})
+
+		name, err := f.FullName()
+		if err != nil {
+			return nil, err
+		}
+
+		if t, ok := f.V.(*core.PdfObjectString); ok {
+			fieldvals = append(fieldvals, fieldValue{
+				Name:  name,
+				Value: t.Decoded(),
+			})
+			continue
+		}
+
+		var val string
+		for _, wa := range f.Annotations {
+			state, found := core.GetName(wa.AS)
+			if found {
+				val = state.String()
+			}
+
+			// Options are the keys in the N/D dictionaries in the AP appearance dict.
+			apDict, has := core.GetDict(wa.AP)
+			if !has {
+				continue
+			}
+			nDict, _ := core.GetDict(apDict.Get("N"))
+			for _, key := range nDict.Keys() {
+				keystr := key.String()
+				if _, has := optMap[keystr]; !has {
+					options = append(options, keystr)
+					optMap[keystr] = struct{}{}
+				}
+			}
+			dDict, _ := core.GetDict(apDict.Get("D"))
+			for _, key := range dDict.Keys() {
+				keystr := key.String()
+				if _, has := optMap[keystr]; !has {
+					options = append(options, keystr)
+					optMap[keystr] = struct{}{}
+				}
+			}
+		}
+
+		fval := fieldValue{
+			Name:    name,
+			Value:   val,
+			Options: options,
+		}
+		fieldvals = append(fieldvals, fval)
+	}
+
+	fdata := FieldData{
+		values: fieldvals,
+	}
+
+	return &fdata, nil
+}
+
+// LoadFromPDFFile loads form field data from a PDF file.
+func LoadFromPDFFile(filePath string) (*FieldData, error) {
+	f, err := os.Open(filePath)
+	if err != nil {
+		return nil, err
+	}
+	defer f.Close()
+
+	return LoadFromPDF(f)
+}
+
+// JSON returns the field data as a string in JSON format.
+func (fd FieldData) JSON() (string, error) {
+	data, err := json.MarshalIndent(fd.values, "", "    ")
+	return string(data), err
+}
+
+// FieldValues implements model.FieldValueProvider interface.
+func (fd *FieldData) FieldValues() (map[string]core.PdfObject, error) {
+	fvalMap := make(map[string]core.PdfObject)
+	for _, fval := range fd.values {
+		if len(fval.Value) > 0 {
+			fvalMap[fval.Name] = core.MakeString(fval.Value)
+		}
+	}
+	return fvalMap, nil
+}
--- a/pdf/fjson/fielddata_test.go
+++ b/pdf/fjson/fielddata_test.go
@ -0,0 +1,134 @@
+/*
+ * This file is subject to the terms and conditions defined in
+ * file 'LICENSE.md', which is part of this source code package.
+ */
+
+package fjson
+
+import (
+	"bytes"
+	"encoding/json"
+	"os"
+	"strings"
+	"testing"
+
+	"github.com/unidoc/unidoc/pdf/model"
+)
+
+func TestLoadPDFFormData(t *testing.T) {
+	fdata, err := LoadFromPDFFile(`./testdata/basicform.pdf`)
+	if err != nil {
+		t.Fatalf("Error: %v", err)
+	}
+
+	data, err := fdata.JSON()
+	if err != nil {
+		t.Fatalf("Error: %v", err)
+	}
+
+	var fields []struct {
+		Name    string   `json:"name"`
+		Value   string   `json:"value"`
+		Options []string `json:"options"`
+	}
+
+	err = json.Unmarshal([]byte(data), &fields)
+	if err != nil {
+		t.Fatalf("Error: %v", err)
+	}
+
+	if len(fields) != 9 {
+		t.Fatalf("Should have 9 fields")
+	}
+
+	// Check first field.
+	if fields[0].Name != "full_name" {
+		t.Fatalf("Incorrect field name (got %s)", fields[0].Name)
+	}
+	if fields[0].Value != "" {
+		t.Fatalf("Value not empty")
+	}
+	if len(fields[0].Options) != 0 {
+		t.Fatalf("Options not empty")
+	}
+
+	// Check another field.
+	if fields[7].Name != "female" {
+		t.Fatalf("Incorrect field name (got %s)", fields[7].Name)
+	}
+	if fields[7].Value != "Off" {
+		t.Fatalf("Value not Off (got %s)", fields[7].Value)
+	}
+	if strings.Join(fields[7].Options, ", ") != "Off, Yes" {
+		t.Fatalf("Wrong options (got %#v)", fields[7].Options)
+	}
+}
+
+// Tests loading JSON form data, filling in a form and loading the PDF form data and comparing the results.
+func TestFillPDFForm(t *testing.T) {
+	fdata, err := LoadFromJSONFile(`./testdata/formdata.json`)
+	if err != nil {
+		t.Fatalf("Error: %v", err)
+	}
+	data, err := fdata.JSON()
+	if err != nil {
+		t.Fatalf("Error: %v", err)
+	}
+
+	// Open pdf to fill.
+	var pdfReader *model.PdfReader
+	{
+		f, err := os.Open(`./testdata/basicform.pdf`)
+		if err != nil {
+			t.Fatalf("Error: %v", err)
+		}
+		defer f.Close()
+		pdfReader, err = model.NewPdfReader(f)
+		if err != nil {
+			t.Fatalf("Error: %v", err)
+		}
+	}
+
+	err = pdfReader.AcroForm.Fill(fdata)
+	if err != nil {
+		t.Fatalf("Error: %v", err)
+	}
+
+	// Write to buffer.
+	var buf bytes.Buffer
+	{
+		// TODO(gunnsth): Implement a simpler method for populating all pages from a reader.
+		pdfWriter := model.NewPdfWriter()
+		for i := range pdfReader.PageList {
+			err := pdfWriter.AddPage(pdfReader.PageList[i])
+			if err != nil {
+				t.Fatalf("Error: %v", err)
+			}
+		}
+		err := pdfWriter.SetForms(pdfReader.AcroForm)
+		if err != nil {
+			t.Fatalf("Error: %v", err)
+		}
+		err = pdfWriter.Write(&buf)
+		if err != nil {
+			t.Fatalf("Error: %v", err)
+		}
+	}
+
+	bufReader := bytes.NewReader(buf.Bytes())
+
+	// Read back.
+	fdata2, err := LoadFromPDF(bufReader)
+	if err != nil {
+		t.Fatalf("Error: %v", err)
+	}
+
+	data2, err := fdata2.JSON()
+	if err != nil {
+		t.Fatalf("Error: %v", err)
+	}
+
+	if data != data2 {
+		t.Fatalf("%s != %s", data, data2)
+	}
+}
--- a/pdf/fjson/testdata/basicform.pdf
+++ b/pdf/fjson/testdata/basicform.pdf
--- a/pdf/fjson/testdata/formdata.json
+++ b/pdf/fjson/testdata/formdata.json
@ -0,0 +1,46 @@
+[
+    {
+        "name": "full_name",
+        "value": "Jónas Þorgrímsson"
+    },
+    {
+        "name": "address_line_1",
+        "value": "Laugalæk 103"
+    },
+    {
+        "name": "address_line_2",
+        "value": ""
+    },
+    {
+        "name": "age",
+        "value": "39"
+    },
+    {
+        "name": "city",
+        "value": "Reykjavík"
+    },
+    {
+        "name": "country",
+        "value": "Ísland"
+    },
+    {
+        "name": "male",
+        "value": "Yes",
+        "options": [
+            "Off",
+            "Yes"
+        ]
+    },
+    {
+        "name": "female",
+        "value": "Off",
+        "options": [
+            "Off",
+            "Yes"
+        ]
+    },
+    {
+        "name": "fav_color",
+        "value": ""
+    }
+]
--- a/pdf/model/colorspace.go
+++ b/pdf/model/colorspace.go
--- a/pdf/model/const.go
+++ b/pdf/model/const.go
@ -9,10 +9,13 @@ import (
 	"errors"
 )

+// Errors when parsing/loading data in PDF.
+// TODO(gunnsth): Unexport errors.
 var (
-	ErrTypeCheck                = errors.New("type check")
 	ErrRequiredAttributeMissing = errors.New("required attribute missing")
 	ErrInvalidAttribute         = errors.New("invalid attribute")
+	ErrTypeCheck                = errors.New("type check")
+	errRangeError               = errors.New("range check error")
 	ErrEncrypted                = errors.New("file needs to be decrypted first")
 	ErrNoFont                   = errors.New("font not defined")
 	ErrFontNotSupported         = errors.New("unsupported font")