Page Comparison

There is often a requirement to work with files using FTP and SFTP. This article highlights the Actions that can be used with these two protocols and how metadata connections are created and used.

The FTP,

...

FTPS and SFTP Protocols

The following protocols exist to transfer files on a TCP/IP network.

• The File Transfer Protocol (FTP) is a standard network protocol used to transfer computer files from one host to another host

...

• .
• FTPS (also known as FTPES, FTP-SSL and FTP Secure) is an extension to the commonly used File Transfer Protocol (FTP) that adds support for the Transport Layer Security (TLS) and the Secure Sockets Layer (SSL) cryptographic protocols.
• The SSH File Transfer Protocol (also Secure File Transfer Protocol, or SFTP) is a network protocol that enables secure file transfer and file management capabilities between networked hosts

...

• .

...

• SFTP uses SSH2 encryption to authenticate and establish secure channels between networked hosts. Although SFTP clients are functionally similar to FTP clients, they employ different protocols and therefore, you cannot use a standard FTP client to connect to an SFTP server.

...

• If you need to create a FTP server on EC2, here are stack overflow instruction to do it. 

Convergence for Data Integration Actions

Convergence for Data Integration provide the following actions to handle files these protocols:

...

• FTP and FTPS
  • Get Files with FTP
  - FtpGet
  • Send Files with FTP
  - FtpPut
  • Command with FTP
  – Launches a script on the FTP Server

...

• SFTP

...

• • Get Files with SCP
  • Send Files with SCP
  SSH – runs a
  • Execute SSH (to runs an SSH script on a remote host)

...

FTP/S and SFTP

...

Metadata

...

Connections

Metadata defined for an FTP/FTPS or an SFTP (SSH) connection is not designed to be used in the same way as reverse engineering a flat file or database metadata. It is only there so you can set up a metadata connection containing used to contain the connection information to the server and directories. It is designed like this so that if you are using its directories. You use such a metadata definition in FTP or SCP actions (nodes) in Processes, you can just drag and drop the metadata connection information, rather than having to type it in manually.

It is not designed to enable you to connect and reverse engineer files directly through FTP/SFTP. FTP/SFTP are transfer protocols, not really designed for manipulating files (or permanently connecting to files). This is why you have specific actions for FTPGet and FTPPut which take care of manipulating the files through that connection.

To reverse engineer files, you need to have a mapped drive to processes, by dragging and dropping it on the action to create a metadata link. By doing so, you configure the action (host, port, user, password, etc.) with the parameters set in the metadata. You cannot reverse-engineer file structure through FTP/SFTP, as they are transfer protocols.

To reverse engineer files stored in a remote host, you should have a drive mapped to the remote location (folder) where those files are stored and use the normal regular File metadata reverse. If you are using both remote and local runtime engines, you would need a configuration for your remote server (the path from the remote runtime on the remote server to the remote location (these may not be the same)) located in the remote location) and another one for your local runtime (the path from your local machine to the remote location).

Working with SFTP

To Use SFTP in Convergence for

...

Data Integrator:

1. Create a an SSH metadata Metadata connection including a the remote folder:.
   Image Modified

...

1. In the Process where the SFTP connection is required, drag and drop the SSH Metadata folder

...

1. on to the SCP action

...

Make
1. to create a metadata link to this folder and make sure you select
“Scp
2. the Scp Use
Sftp”
3. Sftp option in the properties
   Image Modified

...

...

...

Working with FTP or FTPS

To use FTP or

...

FTPS in Convergence for

...

Data Integrator

1. Create an FTP metadata connection including a folder:Metadata Connection including the remote folder
   Image Modified

...

1. In the Process where the FTP connection is

...

1. used, drag and drop the folder

...

1. on to the FTP

...

1. action to create a metadata link to this folder.
   Image Modified

Related articles