Documents
HomeAcademyContactInstallation Guide
User Manual
User Manual
  • Introduction & High Level Architecture
  • 200 OK Versions
    • Prerequisites
      • Internal Authentication Setup
      • Configuration for Email Threshold
    • Release Notes 200 OK Version 5.68.0
    • Release Notes 200 OK Version 5.36.0
    • Release Notes 200 OK Version 5.0.0
    • Release Notes 200 OK Version 4.26.0
    • Release Notes 200 OK Version 4.22.0
    • Release Notes 200 OK Version 4.6.0
    • Release Notes 200 OK Version 4.1.0
    • Release Notes 200 OK Version 3.47.0
    • Release Notes 200 OK Version 3.41.0
  • 200 OK UI Walkthrough
  • Access Configuration
  • Integration Channel
    • How to Create an Integration Channel.
    • Run Your Integration Channel: Let’s Make It Happen!
  • Integration Actions
    • Data Store
    • sObject
    • Apex Class
    • File
    • Platform Event
    • Quick Action
    • Global Action
    • Big Object
    • External Object
    • Integration Channel
    • Auto-launched Flow
    • Callback (client callback)
    • Custom Settings
    • Custom Metadata Type
    • Custom Label
    • Push Notification
  • ETL
    • Create Your ETL
      • Data Source
      • Transform
      • Load
      • Additional Features of ETL
  • Webhook
    • Setting Up Your Webhook!
    • Create Your Webhook!
    • Putting Your Webhook to the Test!
  • Invoke Integration Channel
    • Flow into Action: Invoking Your Integration Channel!
    • Unleash the Power: Invoke with Apex!
  • LWS
    • Transform Data on Demand: ETL as an API!
    • Supercharge Your Workflow: Invoke Integration Channel via API!
  • Built-in Functions
    • Text
      • getChar
      • extract
      • proper
      • len
      • trim
      • rightPad
      • leftPad
      • left
      • right
      • mid
      • getCharIndex
      • coalesce
      • toBoolean
      • toDecimal
      • split
      • replace
      • remove
      • toLowerCase
      • ascii
      • begins
      • capitalize
      • contains
      • deDuplicate
      • escapeHtml
      • generateRandomString
      • getValueFromResponseHeader
      • join
      • reverse
      • startCase
      • stripHtml
      • toBinaryBase64
      • toInt
      • toString
      • toUpperCase
      • escapeMarkDown
    • Math
      • abs
      • acos
      • asin
      • average
      • base64Decode
      • base64Encode
      • ceil
      • generateRandom
      • max
      • min
      • parseNumber
      • round
      • sum
      • switch
      • floor
    • Date & Time
      • toSeconds
      • daysBetween
      • dayPrecision
      • setHour
      • daysInMonth
      • dateTimeNow
      • netWorkDays
      • dateAdd
      • dateDiff
      • addMonths
      • date
      • dateTimeValue
      • dateValue
      • day
      • dayofYear
      • formatDate
      • formatDateToday
      • formatDateTodayFormat
      • formatDateTodayUnix
      • formatDateUnix
      • formatDateUnixFormat
      • toTime
      • toDate
      • toDateTime
      • toDateTimeUnix
    • Advanced
      • generateUUID
      • uncompress
      • compress
      • exists
      • dwTransform
      • discard
      • currencyRate
      • decodeUrl
      • encodeUrl
      • getChannelData
      • getLink
      • invokeFlow
      • invokeETL
      • lookup
      • md5
      • sha1
      • sha256
      • sha512
      • toJson
    • Logical
      • and
      • blankValue
      • case
      • if
      • isBlank
      • isNull
      • isNumber
      • not
      • nullValue
      • or
  • Custom Function
  • Filters for Integration Action
  • Data Mapping Types
    • Key/Value
    • JSON
  • Global variables for Data Mapping
    • Index
    • Record field
    • User
    • Organization
    • Custom Settings
    • Custom Metadata Type(MDT)
  • Advance Setting
  • Schedule
  • Utilities
    • Add Remote Site
    • Json To Apex
    • JSON parser invocable method
    • Export/Import Functionality
    • Import from Postman
    • Curl Import
    • Apex Class Template Creation for Header, Request and Response.
  • Logging
  • Monitor
    • Measure and track the usage
    • Integration channel (total active channels)
    • Callouts (total monthly callouts)
  • Most frequently used scenarios
    • Efficient Bulk Data Handling Using "!index"
    • How to access authentication data from a custom setting?
    • How to set up auth provider and named credentials?
    • How to use named credentials in an integration channel?
    • How to invoke integration channels from flow?
      • Screen flow
      • Record trigger flow
      • Platform event
      • Scheduled flow
    • How to set up a quick action with the integration channel flow?
Powered by GitBook
On this page
  1. ETL
  2. Create Your ETL

Data Source

In the extract phase of ETL, data is collected from various sources, which could include databases, APIs, web services, cloud storage, streaming platforms, and other data repositories.

PreviousCreate Your ETLNextTransform

Last updated 5 months ago

The Extract of ETL starts with selecting the required Data Sources.

There are various Data Sources from which we can extract.

Object

On selecting an Object as Data source, it enables us to select the SObject from the list of SObjects (Standard / Custom) from our Org. It also asks for the fields from the selected SObject from which the data needs to be extracted. Note: Data Source Name can be customized.

Query

The Data Source as SOQL Enables us to write our custom SOQL Query which will query all the data from the current org.

This also has the capability to query the Standard and Custom Objects.

Rest Service

This Rest Service as a Data Source helps us to get the required data using the Rest Service Api which can be used further. Note : To use this functionality the user needs to Set up the current Org Named Credential following this the user needs to create the Custom Setting Record for App Settings and set the Api Name of the created Named Credential in the Current Org Named Credential option.

Apex Class

Data Source as Apex Class enables the feature to write custom logic in Apex Classes and get the return from the apex class.

The apex class drop down lists all those classes which implement the IICDataSourceGenerator class.In the class there must be a method named getData having two parameters (String recordId, String input) with a return type as String. Values to the parameters can be passed using the Parameters input.

For successful usage the class the above method should return a serialized Map.

Custom JSON

This feature helps to add Custom Json will be one of the Data Source in the Extract Part.

External Service

The External Data Source feature helps the ETL to extract the response data from an existing integration channel after making a callout. To make this possible the Integration Channel must be active. The Channel Name section lists out all the Integration channels.

Data Sources

This section of the Extract helps to select the priory created Data Sources.

As we wrap up our exploration of the Extract phase in this section, the spotlight now shifts to the transformative realm. Join us in the next section as we delve into the intricacies of the Transform stage, unraveling the power of data metamorphosis and its pivotal role in shaping a comprehensive ETL process.